Puffin automation framework at a glance

The entire Puffin Automation Framework rests on four types of objects: actions, action tokens, tasks and plans. Let's start with a brief definition of each:

• An action represents an execution item of some kind that may take one or more input, produce one or more output and whose results can be validated against an expected standard of some kind. Actions are the building blocks of tasks.
  
  
• An action token represents a value used as an input for an action or generated as an output from the execution of an action. Action tokens can be persisted to a file system or a database and can be used to control task execution as part of a conditional (do such-and-such until this output token is empty, for example). Action tokens represent the actual data passed around in the execution of Puffin entities.
  
  
• A task is a transactional group of actions that must all succeed for the task to succeed. You can set a task to be executed only if another task succeeds. You can specify that a task, if it fails, should cause the remaining plan to be aborted. A task is the key building block of a plan.
  
  
• A plan is a structured execution scheme for a series of tasks. In a plan, you can specify that a task or group of tasks should be executed only if a condition holds true. You can specify that a task or group of tasks should be repeatedly executed until some condition is true. Finally, it is in your plan that you can specify what action tokens are to be persisted and how.

The Puffin Automation Framework is the "glue" that ties all the above entities together. Puffin evaluates and sets the value of an input (potentially from the output from an earlier action) executing the action. Puffin validates the results of that action's execution, extracts outputs from the action's execution, and finally, on a macro-scale, manages the combination of these entities into tasks and plans.

Also, remember that Puffin is a "framework," which, for Puffin, means that it handles for you all the complexity of automating these execution items. This management can come in two forms: either as a standalone process managed through XML-based definition and configuration files (more on this later) or as a foundation from which your own applications can extend. These concepts will make increasingly more sense as we discuss the various entities in depth below.

Actions and action tokens

The action entity is key to the entire framework. Without actions, the framework would be meaningless. Actions are the most basic building block for interacting with the Puffin Automation Framework and, at the same time, the most customizable portion of the whole framework.

As we discussed above, an action is an executable item of work that may take one or more inputs, may produce one or more outputs, and whose results may be validated against a known standard.

    
<actionConfig>

   <actionList>

     <action name="setLoginInfo" type="Local"/>
   </actionList>
</actionConfig>

In Listing 1, action is identified by the name setLoginInfo and is of type Local. The first is a unique identifier for this action among all those defined for this workspace (more on workspaces later in this article). The second tells the framework that when this action is executed, no communication to another machine will occur. This action stays local to the box. This type of action is used mostly to generate values based on some processing or to prime the framework for other actions used later in a plan.

As it sits right now, this action doesn't do anything. The framework will accept it as part of a plan and execute it at the proper time, but nothing will happen. This action does not produce any outputs. It doesn't take any inputs. Also, as it sits right now, this action is always successful, as it defines no method of validating its results.

Let's change this action's definition to make it generate an output value (Listing 2):

    
<actionConfig>
   <actionList>
      <action name="setLoginInfo" type="Local">
         <output name="USER_NAME" processor="Value">
            <param name="value">keyton</param>


         </output>
      </action>
   </actionList>
</actionConfig>

Now the setLoginInfo action produces a single output, USER_NAME. The value of this output is always the string keyton. There are a couple of things you need to note about outputs. All outputs have a name (USER_NAME, in this case), and all outputs have a processor (Value, in this case). The name, as you would guess, initializes an action token (more on these in just a second) with a unique name by which that value can be used in a subsequent action.

The processor attribute for the output tells the framework how to generate this value. In this case, we've specified that Puffin should just use the value specified in the parameter. Other processors allow for the extraction of a value from an action's results (the HTML returned from a Web action, for example) using a regular expression, from an HTTP header list, or from execution of an XPath expression, against the action's results, etc.

Some processors do such things as generate random numbers or strings. You can define your own custom processors to do pretty much anything you want. What about the data types for all these things? In its current iteration (as it is written using Python), Puffin handles all that for you, but in the future you may be able to specify that, too, should there be a need.

Since the framework isn't doing anything to process this action's execution, there's really no need to specify a method of validation for this action's results (though you certainly could). Also, because this action doesn't really do anything, there's no need to specify any inputs.

In a moment I will delve a bit deeper into actions and what they are. But first, I will take a brief detour and discuss action tokens. Remember that action tokens can be used either as inputs for an action or are produced as outputs from an action.

For the duration of a plan's execution, Puffin maintains a dictionary (also known as a hash) of token names and their current values. Any action can generate (or modify) one of these action tokens by way of an output or use one of them as an input. Although these tokens can change very rapidly over the course of a plan's execution, you also have the ability to persist token values using a token writer, but that is more detail than we need for now. For now, recognize that a token is a named value and that Puffin maintains the current values for all these tokens in a token dictionary.

What about lists/collections? No problem. An action token can correspond to a list of values. Puffin will handle those as well. There are even special list access processors that allow you to access each value in a token list one after another, cycling back to the first once you have retrieved the last value in the list and the like.

Enough about tokens for now. Let's continue discussing actions. Listing 3 is an example of an Http action, which, as you might guess, represents a call to a Web page of some kind. This action has some inputs and an output to provide a more realistic example.

    
<actionConfig>
   <actionList>
      <action name="login" type="Http">
         <actionParams>
            <param name="path">/start.cgi</param>


            <param name="server">demo.puffinhome.org</param>
            <param name="port">80</param>
         </actionParams>
         <input type="GET" name="username" processor="Dict">

            <param name="key">USER_NAME</param>

         </input>
         <input type="GET" name="password" processor="Value">
            <param name="value">puffin</param>

         </input>
         <output name="USER_ID" processor="ExtractRegex">

            <param name="expr" eval="0">
               <![CDATA[href="itemList\.cgi\?user_id=(\d*)]]>
            </param>
         </output>

      </action>
   </actionList>

</actionConfig>

Now this is more like it. The login action defined above is an Http action (it represents an "Http" call to a Web page). The definition starts with some action parameters (server, port, and path to the Web page being called). These tell Puffin how to reach the Web page being called.

Next are two inputs. Puffin will evaluate these and use them in executing the action. How Puffin does this depends on the type of the input (the type attribute) and the type of action (Http in this case) that is being executed. In this case, Puffin will get two values called username and password and will add these values to the querystring (the GET type means HTTP GET in this case) when the Web page is called.

But what values will be used for the username and password inputs? The password is easy. You saw earlier how the Value action token processor works with our Local action. In that case, Puffin will simply get the value from the Value parameter and use that as the input.

The input for the username input is a little different. The action token processor for that input is Dict. As you may be able to guess, this instructs Puffin to look up the current value of the action token whose name is specified by the key parameter in the current token dictionary for this plan's execution and use that value as the value for the input. In this case, that means that Puffin will look up the current value for the USER_NAME action token we initialized earlier and use that value as the value of the username input for this action.

Puffin puts all this information together and makes a Web call -- behind the scenes. That Web page call would look something like this were you to type it into the address box for your Web browser:

  http://demo.puffinhome.org:80/start.cgi?username=keyton&password=puffin

So Puffin has created the above Web page request, executed it, and retrieved its result, whatever that is. Now what? The action definition above tells Puffin that once it has received this action's execution result (an HTML document in this instance), it should look for a user id using the following regular expression (utilizing the ExtractRegex action token processor):

  href="itemList\.cgi\?user_id=(\d*)

Once extracted, Puffin should store this value in an action token called USER_ID in the current token dictionary for this plan execution. This action token, can now, in turn, be used as an input in subsequent actions.

There is one last thing you want to do with this action definition. You want to tell Puffin how this action's results should be validated. You can do this with a validator element, as shown in Listing 4:

    
<actionConfig>
   <actionList>


      <action name="login" type="Http">
         <actionParams>
            <param name="path">/start.cgi</param>
            <param name="server">demo.puffinhome.org</param>

            <param name="port">80</param>

         </actionParams>
         <input type="GET" name="username" processor="Dict">
            <param name="key">USER_NAME</param>

         </input>
         <input type="GET" name="password" processor="Value">

            <param name="value">puffin</param>
         </input>
         <validator type="StatusCode">

            <param name="validStatus">200</param>
            <param name="validStatus">302</param>

         </validator>
         <output name="USER_ID" processor="ExtractRegex">

            <param name="expr" eval="0">
               <![CDATA[href="itemList\.cgi\?user_id=(\d*)]]>
            </param>
         </output>

      </action>
   </actionList>

</actionConfig>

The addition tells Puffin to validate the action's results with a StatusCode validator. This validator is one of several that are included with the Puffin framework core. As with pretty much everything else in Puffin, you can extend the framework with your own custom validators very easily.

In this example, the validator, only useful for Http actions, looks at the code returned by the Web page call and makes sure that code is either 200 or 302. You could also specify one or more invalidStatus parameters (404, for example) instead of one or more valid statuses, if you wanted to.

The login action will be considered successful only if the Http status code returned by the server is 200 or 302. With anything else being returned as the Http status code, the action would have been marked as a failure.

There are several validators that come with Puffin including validators that time an action's execution or evaluate expressions containing action token dictionary entries.

That's a huge amount of information -- especially in XML -- involved in the definition of a single "action" for this framework. Don't worry. You can simplify things greatly. Puffin allows you to specify such things as inputs that should be used for every action (the Http "host" header is a good example of an input you will want to deal with like this), default action parameters, etc.

Before leaving the discussion of actions, note that ALL actions are defined using the exact same format in the action.xml file for the workspace (see below) with which you are working. You may have actions that connect to a database, listen to a socket, or open a file. Though each may have different types of inputs or validators, they would all be defined exactly the same way as the login action above.

To conclude the discussion of actions, let's look at a database action example:

      <action name="getSalesByUser" type="MySql">

         <actionParams>
            <param name="host">mysql.puffinhome.org</param>
            <param name="user">keyton</param>

            <param name="pass">puffin</param>

            <param name="db">sales</param>
         </actionParams>
         <input type="DML" name="salesQuery" processor="Value">

            <param name="value">
                  SELECT * 
                    FROM sales 
                   WHERE user_id = "$$$USER_ID$$$"
            </param>

         </input>
         <validator type="ResultCount">
            <param name="min">1</param>

         </validator>
         <output name="SALES_RESULTS" processor="DataList"/>

         </output>
      </action>

The above action definition demonstrates lots of the same ideas as the earlier login action example. One new concept is the use of a token expression. In almost every situation where a value could be used (and that value is known at runtime), you can use a token expression, which is just a token name prefixed and suffixed with a $$$. In this case, the token expression is $$$USER_NAME$$$. This instructs Puffin to replace that token with the current value of the USER_NAME token. (So why have both a Dict and a Value processor, you may be wondering? History and flexibility are the only answers.)

Now you have a pretty good idea of the basics of actions. This is the most important concept to understand in the entire Puffin Automation Framework, since actions are the basic building blocks upon which everything else is built.

You can see how the setLoginInfo, login, and getSalesByUser actions fit together. The USER_NAME output from setLoginInfo is used as the username input for login. The USER_ID output for login is part of the input used by getSalesByUser. Puffin handles all of this for you. You don't need to understand (or even care) about the underlying details. Puffin handles all of these complex communications and the data maintenance among them.