Installing Puffin

Fortunately, installing Puffin is fairly simple (see Resources later in this article for links to each download):

1. Download and install the appropriate Python distribution for your platform.
2. Download and install the PyXML package for your platform.
3. Download and install the eGenix base package of Python extensions.
4. Download and uncompress Puffin.

In the "puffin/doc" folder are two documents: "GettingStarted.htm" and "UserGuide.htm." Read through the Getting Started document to see how to set up the test application (used as our demo below). Tackle the User Guide after you have finished this article.

From this point on, we will assume that you have Puffin installed, but have not necessarily run anything yet.

Framework configuration

Before you can start writing tests, you must configure the Puffin framework to your Web application. You do this through a single XML file called (by default -- this can be overridden) puffinConfig.xml. In this file, we will set up the following aspects of the Puffin system:

• Address of our Web application's server
• Logging channels for the framework itself and results processing
• Level of detail desired for the results reporting
• Universal response analyzers for checking the validity of all responses in your Web application (we'll talk about these in a moment)
• Possible test actions callable against the Web application

For this example, we will create a config file from scratch, though you will likely just customize the one that comes with Puffin. For right now, rename the puffinConfig.xml file that comes with Puffin to puffinConfigORIG.xml or similar and create a new text file called puffinConfig.xml. Add the following XML to it, and save it in the same folder where puffin.py is located:

   <puffin>
      <system/>

      <testActions/>
   </puffin>

As you can see, the config file has two main sections. The first describes the various properties of the Web application system itself. The second describes the possible test actions you can call in the test plans you will create.

The first system element we need to specify is the server information. This is as simple as it sounds. Just add a <server> element with the location of your Web application server:

   <puffin>
      <system>

         <server host="www.mydomain.com" port="80"/>
      </system>
      <testActions/>
   </puffin>

The <server> element, as you can see, has two attributes:

• The host attribute specifies a host name.
• The port attribute specifies a port number.

Note that we will assume you set up the demo application locally and will thus set the server element accordingly (though this isn't strictly necessary as localhost and port 80 are the defaults if no <server> element is in place):

   <puffin>
      <system>

         <server host="localhost" port="80"/>
      </system>
      <testActions/>
   </puffin>

Puffin knows where to find your Web application's server. Now we need to set up how Puffin will tell you what is happening with your test plans as they execute. There are three types of information that Puffin will attempt to relay to you:

• A log of the activity of the Puffin framework itself
• A report describing the results of a given test plan's execution
• A "mini-report" of a failed task occurring during the execution of a test plan

Puffin uses (and dare I say "abuses") a logging system based on the popular log4j design by Vinay Sajip (see Resources) to present this information. If you are familiar with log4j or log4py or pretty much any log4* product, then you will already understand logging priorities and logging handlers and can skip the next two paragraphs.

The logging system used in Puffin allows you to set a priority threshold for each channel of logging. So for example, if I set the priority of the logging channel that handles messages from the framework itself to a level of WARN, then only messages sent with WARN or above will actually get logged. All others will be ignored. The priority levels are DEBUG, INFO, WARN, ERROR, and FATAL.

Handlers are used to direct logging statements to targets. You may have a handler that will direct your logging statements to the command line, or to an e-mail address, or (on Windows NT) to the event log. You can stack handlers too. For example, I could set it up so that all failed task "logging" messages for Puffin go to an e-mail address, and all framework logging messages go to the console and to a file.

As you might guess, you configure the Puffin logging channels through the exact same config XML file with which we have been working. Here are three more entries:

   <puffin>

      <system>
         <server>localhost</server>

         <frameworkLogging channelPriority="WARN">
            <handler type="StreamHandler">
               <param name="msgFormat"><![CDATA[%(asctime)s %(name)s %(message)s]]></param>

            </handler>
         </frameworkLogging>

         <reportLogging channelPriority="WARN">
            <handler type="FileHandler">
               <param name="msgFormat" eval="0"><![CDATA[%(message)s]]></param>

               <param name="fileName" eval="0">puffinResults.log</param>
               <param name="mode" eval="0"><![CDATA[a+]]></param>

            </handler>
         </reportLogging>

         <failureAlertLogging channelPriority="WARN">
            <handler type="FileHandler">
               <param name="msgFormat" eval="0"<![CDATA[%(message)s]]></param>

               <param name="fileName" eval="0">puffinFailures.log</param>

               <param name="mode" eval="0"><![CDATA[a+]]>
               </param>
            </handler>
         </failureAlertLogging>

      </system>

      <testActions/>
   </puffin>

The three logging entries allow you to set the priority and set one or more handlers. The above sets up the StreamHandler (console logging) for the framework itself and file handlers for both the results reporting and the failed tasks alerts. See the Puffin User's Guide for all the handlers supported. The key attribute is the priority. All report and failure information is logged at a level of WARN. If you do not set these priorities to at least this level, no reports or failure alerts will be written. The framework itself does some logging at WARN (mostly just letting you know status) and some at higher (ERROR or FATAL) levels. It also has LOTS of debugging in place as we are still only at a 0.8 release!

Puffin now knows where your server is and how to communicate to you. Now we need to express how much information for test plan execution results you want Puffin to communicate. We do this with a single additional line in the config file:

<puffin>
   <system>
      <server>localhost</server>

      <frameworkLogging channelPriority="WARN">
         <handler type="StreamHandler">
            <param name="msgFormat"><![CDATA[%(asctime)s %(name)s%(message)s]]></param>
         </handler>
      </frameworkLogging>

      <reportLogging channelPriority="WARN">
         <handler type="FileHandler">
            <param name="msgFormat" eval="0"><![CDATA[%(message)s]]></param>
            <param name="fileName" eval="0">puffinResults.log</param>

            <param name="mode" eval="0"><![CDATA[a+]]></param>

         </handler>
      </reportLogging>
      <failureAlertLogging channelPriority="WARN">
         <handler type="FileHandler">

            <param name="msgFormat" eval="0"><![CDATA[%(message)s]]></param>

            <param name="fileName" eval="0">puffinFailures.log</param>
            <param name="mode" eval="0"><![CDATA[a+]]></param>
         </handler>

      </failureAlertLogging>
      <resultsProcessor reportDetail="ALL"/>

   </system>
   <testActions/>
</puffin>

For now, we'll leave the detail level at "ALL," which means that you will get back more than you probably want for results of a test plan's execution. But that's okay for learning the system. See the User's Guide if you want to trim it back a bit.

Before we complete our configuration (we only have one more entry for now), we need to take a break to discuss how Puffin works in "testing" your Web application.