12 Commits

Author SHA1 Message Date
Phanes
1c354b4401 updated readme to be more clear 2017-12-08 00:34:29 -05:00
Phanes
0db575a075 Merge remote-tracking branch 'origin/master' 2017-12-07 22:38:05 -05:00
Phanes
46e56b8d6f implemented execution context. needs thoroughly tested. 2017-12-07 22:37:42 -05:00
Chris Punches
93c22887e8 Update README.md 2017-12-07 10:07:04 +00:00
Phanes
9d89f5ad6a minor punctuation fixes 2017-12-07 02:39:27 -05:00
Phanes
6fca4f2d91 added markdown extention 2017-12-07 02:32:56 -05:00
Phanes
c75fa9d2c8 updated readme 2017-12-07 02:24:17 -05:00
Phanes
49b4e77ca8 updated readme 2017-12-07 02:05:03 -05:00
Phanes
8a95bbf219 not sure why but ok 2017-12-07 01:32:00 -05:00
Chris Punches
4d66a0f059 Merge branch 'master' into 'master'
Honor long options as well.

See merge request !1
2017-12-07 06:22:55 +00:00
DJ Lucas
221edee07c Forgot print_usage() on invalid option. 2017-12-05 22:40:57 -06:00
DJ Lucas
0c70f7ee36 Honor long options as well. 2017-12-05 22:09:33 -06:00
11 changed files with 340 additions and 35 deletions

123
README.md Normal file
View File

@@ -0,0 +1,123 @@
# Instructions
These are instructions for using Examplar.
## Build
Compiling Examplar is easy. There are zero external dependencies. Build does require *cmake*.
~~~~
$ cmake .
$ make
~~~~
Then place the binary where you'd like. I'd recommend packaging it for your favorite Linux distribution.
## High Level Usage
1. Write a script that does a thing.
2. Write a script that checks if that thing is done.
3. Set up your check as a target in a unit.
4. Set up your script as its rectifier.
5. Turn on the rectify pattern in the unit definition.
## Definitions
So you've got Examplar compiled and you're ready to start automating the world.
If you're thinking "how do I configure this thing", this article is for you.
### Units
A Unit is an automation definition, written in JSON in a UNIT FILE. Deeper into Examplar’s internals, Units and Tasks have slightly different functions, but for the purposes of users, the terms can be used interchangeably. A Task is a task to be performed in a Plan, and a Unit is its definition. A Unit is a JSON object that has:
* A `name`, which is an identifier for the Unit used by people.
* A `target`, which is the path to the automation script performing the work. This provides a clean linear path for huge chains of scripts to be executed in order and tracked on return for additional logic in chaining.
* A `rectifier`, which is the path to the automation script to be executed if the target call fails.
* A `rectify` attribute, which tells Examplar whether or not to execute the rectifier in the case of failure when executing the target.
* An `active` attribute,which tells Examplar whether or not the Unit can be used in a Plan. This gives Unit developers a way to tell Plan developers not to use the Unit.
* A `required` attribute which tells Examplar whether or not the Plan can continue if the Unit fails. If the rectify attribute is set to true, this attribute is checked after a rectifier failure. If not, this is checked after target failure. In either case, if the rectifier or target do not return successfully, Examplar will halt the execution of the Plan if this is turned on for the unit being executed. Otherwise it simply moves to the next Unit being executed.
### Tasks
A Task is an action item in a Plan, just like in real life. In the context of Examplar, a Task is a Unit that has been loaded and incorporated into a Plan in an actionable state. Inactive Units can not be loaded into a Plan and thus can never be a Task. The primary difference between a Task and a Unit is that a Unit is not actionable — it’s just a definition — while a Task a consumable, actionable automation.
Suite
A Suite is not visible to the user and this is only for informational purposes. A Suite is a collection of all available Unit definitions loaded from one or more UNIT FILES. Just as a Unit is the definition for a Task, a Suite is a collection of Units that define the Task components of a Plan.
A Suite is consumed by a Plan during the conversion of Units to Tasks, though this is not visible to the user — it just simply helps to understand the kind of abstraction taking place in the conceptual model of Examplar.
Plan
A Plan is the glue of all the components of Examplar and is deceptively simple. A Plan loads a Suite for its Task definitions (Units), but the Tasks to actually execute are specified in the PLAN FILE. The Tasks are executed in the order specified in the PLAN FILE.
### FILES
There are several files used by Examplar.
#### CONFIG FILE and Attributes
This is the one config file that Examplar uses. The default path it looks is /etc/Examplar/config.json.
A config file at the time of writing this specifies a single JSON object with 5 attributes:
* `units_path`: The `UNIT FILE` path or a path to a directory containing unit files.
* `plan_path`: The PLAN FILE path. There is only ever one plan executed in a single run.
* `config_version`: The configuration VERSION.
* `execution_context`: The current working directory to use when loading unit files, plan files, or executing Tasks.
* `execution_context_override`: A boolean indicating whether or not the execution context should be set, or left alone. It is highly recommended to set this to `true`.
#### Configuration VERSION
The configuration version is checked to ensure that the configuration is consumable by that version of Examplar. This will pave the way for reverse compatibility if the project moves in that direction.
#### UNIT FILE
The UNIT FILE is a specification of where the Units are defined. All UNIT FILES in that directory will be amalgamated to generate the Suite. These types of files must end in `*.units` for their filename.
#### PLAN FILE
The PLAN FILE is a specification of the order that Tasks are executed, and their dependencies upon each other. Dependency implementation is a touchy matter that is pending implementation, so, mileage may vary until release.
## I still don't see how this works.
That's ok. It's in its infancy so we're always looking for ways to make it simpler. Here's a 'hello world' example.
### 1. Write your tests.
First, we want to know all the things we need to be able to print "hello world" to the screen. In this case we just need to make we have the "echo" binary.
Write a bash script that checks if the "echo" binary is on the system.
#!/usr/bin/bash
stat /usr/bin/echo
exit $?
Save it as ~/check-echo.bash.
This script will be your "target" attribute for your "hello world" unit definition.
### 2. Write your automation.
Write a "hello world" script.
#!/usr/bin/bash
echo "hello world"
exit $?
Save it as ~/hello.bash
This script will be your "rectify" attribute for your "hello world" unit definition.
### 3. Set up the Unit file.
At this point you've got both the script that checks if hello world can run and you've got your hello world script. Time to set up the unit.
### 4. Add the Unit definition to the Plan.
Next, add the unit to the plan by name.
### 5. Set up your config file.
Point your config file at your plan file and your units directory.
### 6. Run Examplar pointing at that config file.
Execute examplar:
examplar --verbose --config path/to/your/config/file.json
And you should see your 'hello world' script.

View File

@@ -1,5 +1,7 @@
{ {
"units_path": "/home/phanes/development/internal/Examplar/conf/units/all_test.units", "execution_context_override": true,
"plan_path": "/home/phanes/development/internal/Examplar/conf/plans/test.plan", "execution_context": "/home/phanes/development/internal/Examplar/conf/",
"config_version": "1" "units_path": "units/all_test.units",
"plan_path": "plans/test.plan",
"config_version": "2"
} }

View File

@@ -22,8 +22,9 @@
#include "src/json/json.h" #include "src/json/json.h"
#include "src/loaders/loaders.h" #include "src/loaders/loaders.h"
#include <unistd.h> #include <unistd.h>
#include <getopt.h>
#include <syslog.h> #include <syslog.h>
#include "src/loaders/helpers.h"
/* /*
* TODO Commandline switches * TODO Commandline switches
@@ -31,26 +32,54 @@
void print_usage() void print_usage()
{ {
printf("examplar [ -h ] [ -v ] [ -c CONFIG_PATH ]"); printf("examplar [ -h | --help ] [ -v | --verbose ] [ -e | --execution-context EXECUTION_CONTEXT ][ -c | --config CONFIG_PATH ]\n\n");
exit(0);
} }
int main( int argc, char * argv[] ) int main( int argc, char * argv[] )
{ {
int flags, opt; int flags, opt;
bool verbose = false; bool verbose = false;
bool show_help = false; bool show_help = false;
// indicator of whether examplar should use a commandline argument for overriding the context
// instead of what's supplied in the conf file
bool cli_context_supplied = false;
std::string config_path = "/etc/Examplar/config.json"; std::string config_path = "/etc/Examplar/config.json";
std::string execution_context;
// commandline switches: // commandline switches:
// -h help // -h help
// -v verbose // -v verbose
// -c CONFIG_FILE_PATH -- defaults to '/etc/Examplar/config.json' // -c CONFIG_FILE_PATH -- defaults to '/etc/Examplar/config.json'
// -e EXECUTION_CONTEXT -- current working directory when executing unit targets
while ( ( opt = getopt( argc, argv, "hvc:" ) ) != -1 ) while (1)
{ {
static struct option long_options[] =
{
{"verbose", no_argument, 0, 'v'},
{"help", no_argument, 0, 'h'},
{"config", required_argument, 0, 'c'},
{"execution-context", required_argument, 0, 'e'},
{0, 0}
};
int option_index = 0;
opt = getopt_long (argc, argv, "vhec:", long_options, &option_index);
if (opt == -1)
break;
switch (opt) switch (opt)
{ {
case 0:
if (long_options[option_index].flag !=0)
break;
case 'h': case 'h':
show_help = true; show_help = true;
case 'v': case 'v':
@@ -59,6 +88,13 @@ int main( int argc, char * argv[] )
case 'c': case 'c':
config_path = std::string(optarg); config_path = std::string(optarg);
break; break;
case '?':
print_usage();
exit(1);
case 'e':
cli_context_supplied = true;
execution_context = std::string(optarg);
break;
default: default:
break; break;
} }
@@ -67,6 +103,7 @@ int main( int argc, char * argv[] )
if ( show_help == true ) if ( show_help == true )
{ {
print_usage(); print_usage();
exit(0);
} }
setlogmask( LOG_UPTO( LOG_INFO ) ); setlogmask( LOG_UPTO( LOG_INFO ) );
@@ -77,14 +114,33 @@ int main( int argc, char * argv[] )
// A Plan declares what units are executed and a Suite declares the definitions of those units. // A Plan declares what units are executed and a Suite declares the definitions of those units.
Conf configuration = Conf(config_path, verbose ); Conf configuration = Conf(config_path, verbose );
// load the configuration file which contains filepaths to definitions of a plan and definitions of units. // if the user set this option as a commandline argument
if ( cli_context_supplied == true )
{
// override the conf file's specified execution context
configuration.set_execution_context( execution_context );
}
// check if context override
if ( configuration.has_context_override() )
{
// if so, set the CWD.
chdir( configuration.get_execution_context().c_str() );
std::ostringstream infostring;
infostring << "Execution context: " << get_working_path() << std::endl;
syslog(LOG_INFO, infostring.str().c_str() );
std::cout << infostring.str();
}
// load the filepaths to definitions of a plan and definitions of units.
std::string definitions_file = configuration.get_units_path(); std::string definitions_file = configuration.get_units_path();
std::string plan_file = configuration.get_plan_path(); std::string plan_file = configuration.get_plan_path();
Suite available_definitions; Suite available_definitions;
available_definitions.load_units_file( definitions_file, verbose ); available_definitions.load_units_file( definitions_file, verbose );
Plan plan; Plan plan( &configuration );
plan.load_plan_file( plan_file, verbose ); plan.load_plan_file( plan_file, verbose );
plan.load_definitions( available_definitions, verbose ); plan.load_definitions( available_definitions, verbose );

View File

@@ -19,17 +19,48 @@
*/ */
#include "Conf.h" #include "Conf.h"
/// CONF_PLANPATH_INVALID - Exception thrown when the Conf type can not load the supplied path for the Plan definition /// ConfigLoadException - General exception handler for the Conf class.
/// file. class ConfigLoadException: public std::exception
class CONF_PLANPATH_INVALID: public std::runtime_error { public: {
CONF_PLANPATH_INVALID(): std::runtime_error("conf: The supplied path for the plan definition file is invalid.") {} public:
/** Constructor (C strings).
* @param message C-style string error message.
* The string contents are copied upon construction.
* Hence, responsibility for deleting the char* lies
* with the caller.
*/
explicit ConfigLoadException(const char* message):
msg_(message)
{
}
/** Constructor (C++ STL strings).
* @param message The error message.
*/
explicit ConfigLoadException(const std::string& message):
msg_(message)
{}
/** Destructor.
* Virtual to allow for subclassing.
*/
virtual ~ConfigLoadException() throw (){}
/** Returns a pointer to the (constant) error description.
* @return A pointer to a const char*. The underlying memory
* is in posession of the Exception object. Callers must
* not attempt to free the memory.
*/
virtual const char* what() const throw (){
return msg_.c_str();
}
protected:
/** Error message.
*/
std::string msg_;
}; };
/// CONF_UNITSPATH_INVALID - Exception thrown when the Conf type can not load the supplied path for the Unit definition
/// files.
class CONF_UNITSPATH_INVALID: public std::runtime_error { public:
CONF_UNITSPATH_INVALID(): std::runtime_error("conf: The supplied path for the unit definition file is invalid.") {}
};
/// Conf::Conf - Constructor for Conf type. Loads the configuration for the application. /// Conf::Conf - Constructor for Conf type. Loads the configuration for the application.
/// TODO Expand to detect when a directory path is supplied for units_path or plan_path and import all Tasks and Units. /// TODO Expand to detect when a directory path is supplied for units_path or plan_path and import all Tasks and Units.
@@ -37,18 +68,67 @@ class CONF_UNITSPATH_INVALID: public std::runtime_error { public:
/// \param filename - The filename to load the configuration from. /// \param filename - The filename to load the configuration from.
Conf::Conf( std::string filename, bool verbose ): JSON_Loader() Conf::Conf( std::string filename, bool verbose ): JSON_Loader()
{ {
// prepare context spaghetti
this->override_context = false;
// load the conf file. // load the conf file.
this->load_json_file( filename, verbose ); this->load_json_file( filename, verbose );
if (this->get_serialized(this->config_version, "config_version" ,true) != 0)
{
throw ConfigLoadException("config_version string is not set in the config file supplied: " + filename);
}
if ( this->config_version.asString() != VERSION_STRING )
{
throw ConfigLoadException("config_version string expected was " + std::string(VERSION_STRING) + " in: " + filename);
}
// find the path to the plan file // find the path to the plan file
if (this->get_serialized(this->plan_path, "plan_path", true) != 0 ) { throw CONF_PLANPATH_INVALID(); } if (this->get_serialized(this->plan_path, "plan_path", true) != 0 )
{
throw ConfigLoadException("plan_path string is not set in the config file supplied:" + filename);
}
// find the path to the unit definitions file // find the path to the unit definitions file
if (this->get_serialized(this->units_path, "units_path", true) != 0 ) { throw CONF_UNITSPATH_INVALID(); } if (this->get_serialized(this->units_path, "units_path", true) != 0 )
{
throw ConfigLoadException("units_path string is not set in the config file supplied: " + filename);
}
if ( this->get_serialized(this->override_execution_context, "execution_context_override", true) != 0 )
{
throw ConfigLoadException("execution_context_override boolean is not set in the config file supplied: " + filename);
} else {
this->override_context = true;
}
if ( this->get_serialized(this->execution_context, "execution_context", true) != 0 )
{
throw ConfigLoadException("execution_context string is not set in the config file supplied: " + filename);
} else {
this->execution_context_literal = this->execution_context.asString();
}
}; };
/// Conf::has_context_override - Specifies whether or not the override context function is enabled in the conf file.
bool Conf::has_context_override() {
return this->override_execution_context.asBool();
}
/// Conf::get_execution_context - Specifies the path to the current working directory to set for all unit executions.
std::string Conf::get_execution_context() {
return this->execution_context_literal;
}
/// Conf::get_plan_path - Retrieves the path to the Plan definition file from the application configuration file. /// Conf::get_plan_path - Retrieves the path to the Plan definition file from the application configuration file.
std::string Conf::get_plan_path() { return this->plan_path.asString(); } std::string Conf::get_plan_path() { return this->plan_path.asString(); }
/// Conf::get_units_path - Retrieves the path to the Unit definition file from the application configuration file. /// Conf::get_units_path - Retrieves the path to the Unit definition file from the application configuration file.
std::string Conf::get_units_path() { return this->units_path.asString(); } std::string Conf::get_units_path() { return this->units_path.asString(); }
/// Conf::set_execution_context- Sets the execution context.
void Conf::set_execution_context( std::string execution_context )
{
this->execution_context_literal = execution_context;
}

View File

@@ -21,17 +21,40 @@
#ifndef FTESTS_CONF_H #ifndef FTESTS_CONF_H
#define FTESTS_CONF_H #define FTESTS_CONF_H
#include "JSON_Loader.h" #include "JSON_Loader.h"
#include <exception>
#define STRINGIZE2(s) #s
#define STRINGIZE(s) STRINGIZE2(s)
# define IMPL_CONFIG_VERSION 2
# define VERSION_STRING STRINGIZE(IMPL_CONFIG_VERSION)
class Conf: public JSON_Loader class Conf: public JSON_Loader
{ {
private: private:
Json::Value plan_path; Json::Value plan_path;
Json::Value units_path; Json::Value units_path;
Json::Value execution_context;
Json::Value config_version;
// flag to indicate if execution context should be overriden in config file
// if set to true Examplar should use whats in the config file for current working directory
// if set to false, Examplar should use the current working directory at time of execution
Json::Value override_execution_context;
bool override_context;
std::string execution_context_literal;
public: public:
Conf( std::string filename, bool verbose ); Conf( std::string filename, bool verbose );
std::string get_plan_path(); std::string get_plan_path();
std::string get_units_path(); std::string get_units_path();
bool has_context_override();
std::string get_execution_context();
void set_execution_context( std::string );
}; };
#endif //FTESTS_CONF_H #endif //FTESTS_CONF_H

View File

@@ -120,7 +120,10 @@ protected:
/// Plan::Plan() - Constructor for Plan class. A Plan is a managed container for a Task vector. These tasks reference /// Plan::Plan() - Constructor for Plan class. A Plan is a managed container for a Task vector. These tasks reference
/// Units that are defined in the Units files (Suite). If Units are definitions, Tasks are selections of those /// Units that are defined in the Units files (Suite). If Units are definitions, Tasks are selections of those
/// definitions to execute, and if Units together form a Suite, Tasks together form a Plan. /// definitions to execute, and if Units together form a Suite, Tasks together form a Plan.
Plan::Plan(): JSON_Loader() {}; Plan::Plan( Conf * configuration ): JSON_Loader()
{
this->configuration = configuration;
};
/// Plan::load_plan_file - Uses the json_root buffer on each run to append intact Units as they're deserialized from /// Plan::load_plan_file - Uses the json_root buffer on each run to append intact Units as they're deserialized from
/// the provided file. /// the provided file.
@@ -259,7 +262,7 @@ void Plan::execute( bool verbose )
std::cout << "Executing task \"" << this->tasks[i].get_name() << "\"." << std::endl; std::cout << "Executing task \"" << this->tasks[i].get_name() << "\"." << std::endl;
} }
try { try {
this->tasks[i].execute( verbose ); this->tasks[i].execute( this->configuration, verbose );
} }
catch (std::exception& e) { catch (std::exception& e) {
throw Plan_Task_GeneralExecutionException( "Plan Task: \"" + this->tasks[i].get_name() + "\" reported: " + e.what() ); throw Plan_Task_GeneralExecutionException( "Plan Task: \"" + this->tasks[i].get_name() + "\" reported: " + e.what() );

View File

@@ -25,15 +25,17 @@
#include "../json/json.h" #include "../json/json.h"
#include "JSON_Loader.h" #include "JSON_Loader.h"
#include "Task.h" #include "Task.h"
#include "Conf.h"
class Plan: public JSON_Loader class Plan: public JSON_Loader
{ {
private: private:
// storage for the tasks that make up the plan // storage for the tasks that make up the plan
std::vector<Task> tasks; std::vector<Task> tasks;
Conf * configuration;
public: public:
Plan(); Plan( Conf * configuration );
// append this->tasks from JSON file // append this->tasks from JSON file
void load_plan_file( std::string filename, bool verbose ); void load_plan_file( std::string filename, bool verbose );

View File

@@ -19,10 +19,10 @@
*/ */
#include "Task.h" #include "Task.h"
#include <unistd.h>
#include <stdio.h> #include <stdio.h>
#include <syslog.h> #include <syslog.h>
#include "../sproc/Sproc.h" #include "../sproc/Sproc.h"
#include "helpers.h"
/// Task_InvalidDataStructure - Exception thrown when a Task is defined with invalid JSON. /// Task_InvalidDataStructure - Exception thrown when a Task is defined with invalid JSON.
class Task_InvalidDataStructure: public std::runtime_error { class Task_InvalidDataStructure: public std::runtime_error {
@@ -174,15 +174,10 @@ bool Task::has_definition()
return this->defined; return this->defined;
} }
/// Task::execute - execute a task's unit definition. /// Task::execute - execute a task's unit definition.
/// See the design document for what flow control needs to look like here. /// See the design document for what flow control needs to look like here.
/// \param verbose - Verbosity level - not implemented yet. /// \param verbose - Verbosity level - not implemented yet.
void Task::execute( bool verbose ) void Task::execute( Conf * configuration, bool verbose )
{ {
// DUFFING - If Examplar is broken it's probably going to be in this block. // DUFFING - If Examplar is broken it's probably going to be in this block.
// Somebody come clean this up, eh? // Somebody come clean this up, eh?
@@ -199,18 +194,28 @@ void Task::execute( bool verbose )
std::string task_name = this->definition.get_name(); std::string task_name = this->definition.get_name();
// END PREWORK // END PREWORK
// get the target execution command // get the target execution command
std::string target_command = this->definition.get_target(); std::string target_command = this->definition.get_target();
// if we're in verbose mode, do some verbose things // if we're in verbose mode, do some verbose things
if ( verbose ) if ( verbose )
{ {
infostring = std::ostringstream(); infostring = std::ostringstream();
infostring << "\tUsing unit \"" << task_name << "\"." << std::endl; infostring << "\tUsing unit \"" << task_name << "\"." << std::endl;
syslog( LOG_INFO, infostring.str().c_str() ); syslog( LOG_INFO, infostring.str().c_str() );
std::cout << infostring.str(); std::cout << infostring.str();
// check if context override
if ( configuration->has_context_override() )
{
// if so, set the CWD.
chdir( configuration->get_execution_context().c_str() );
infostring = std::ostringstream();
infostring << "\tExecution context: " << get_working_path() << std::endl;
syslog(LOG_INFO, infostring.str().c_str() );
std::cout << infostring.str();
}
infostring = std::ostringstream(); infostring = std::ostringstream();
infostring << "\tExecuting target \"" << target_command << "\"." << std::endl; infostring << "\tExecuting target \"" << target_command << "\"." << std::endl;

View File

@@ -21,9 +21,11 @@
#ifndef FTESTS_TASK_H #ifndef FTESTS_TASK_H
#define FTESTS_TASK_H #define FTESTS_TASK_H
#include <string> #include <string>
#include <unistd.h>
#include "../json/json.h" #include "../json/json.h"
#include "Unit.h" #include "Unit.h"
#include "Suite.h" #include "Suite.h"
#include "Conf.h"
class Task class Task
{ {
@@ -62,7 +64,7 @@ class Task
std::string get_name(); std::string get_name();
// execute this task's definition // execute this task's definition
void execute( bool verbose ); void execute( Conf * configuration, bool verbose );
void mark_complete(); void mark_complete();

View File

@@ -25,3 +25,9 @@ bool exists(const std::string& name)
struct stat buffer; struct stat buffer;
return (stat (name.c_str(), &buffer) == 0); return (stat (name.c_str(), &buffer) == 0);
} }
std::string get_working_path()
{
char temp[MAXPATHLEN];
return ( getcwd(temp, MAXPATHLEN) ? std::string( temp ) : std::string("") );
}

View File

@@ -22,9 +22,12 @@
#define FTESTS_HELPERS_H #define FTESTS_HELPERS_H
#include <string> #include <string>
#include <sys/stat.h> #include <sys/stat.h>
#include <sys/param.h>
#include <unistd.h>
bool exists (const std::string& name); bool exists (const std::string& name);
std::string get_working_path();
#endif //FTESTS_HELPERS_H #endif //FTESTS_HELPERS_H