Response analysis

As we will see in just a little while, Puffin test plans are broken up into a series of test actions. Each test action represents a single request-response interaction with your Web application. We'll get into the details of this interaction in a few moments. For now, we will discuss only the response that results from the execution of a test action. Puffin will analyze this response for criteria you specify and will use the results of that analysis to deem a given test action a success or failure.

For example, if you are running a JSP-based Web application and have set up an error page that says something like "You have experienced an error. Please call support at ...", then Puffin can analyze the HTTP response resulting from a test action and "look" for the phrase "experienced an error". If Puffin finds that phrase upon executing a test action, that test action will be considered a failure. As another example, you will want to simply ensure that a given test action (Web application call) takes no more than a set amount of time or that it always returns the HTTP Status of "200" meaning everything went okay.

Puffin performs these types of analyses using a ResponseAnalyzer object, of which there are several types core to the system (and, of course, you can extend Puffin with your own). You set these up using the config file just like everything else. You can set up universal response analyzers that Puffin will execute against every single test action's response or just for a specific test action. The former type is set up in the <system> element of the config file; the latter is set up with the individual test action.

We will cover test action-specific response analyzers in a later article. Here is how to set up the universal ones (note that the top half of the config file was snipped off for brevity):

            <param name="mode" eval="0"><![CDATA[a+]]></param>

         </handler>

      </failureAlertLogging>
      <resultsProcessor reportDetail="ALL"/>
      <defaultResponseAnalyzerList>
         <responseAnalyzer src="puffin" type="TimerResponseAnalyzer">
            <param name="timeLimitSecs">10</param>

         </responseAnalyzer>
         <responseAnalyzer src="puffin" type="StatusResponseAnalyzer">
            <param name="httpStatus">200</param>
            <param name="httpStatus">302</param>

         </responseAnalyzer>

      </defaultResponseAnalyzerList>
   </system>
   <testActions/>
</puffin>

As you can probably surmise, the above sets up Puffin so that any test action that does not have its own response analyzers will have its response analyzed using these two response analyzers:

• TimerResponseAnalyzer checks to see that the response came back in less than (in this case) 10 seconds.
• StatusResponseAnalyzer checks to see that the server returns either a "200" (for okay) or a "302" (for redirects) as an HTTP Status code.

Now when you run a test plan, the response of every test action (without its own response analyzer) will be checked to see if it came back in the time allowed and that the HTTP status code was 200 or 302. If either of these results in a failure, then Puffin considers that whole test action a failure.

There are some other things you can set up as system wide (flag settings for security being on or off, whether or not to always send a cookie, etc.), but we will leave it here for now. Check the User's Guide for detail on these more advanced features. Now let's dive into the test action configuration. At this point, your puffinConfig.xml file should look like this:

<puffin>
   <system>
      <server>localhost</server>

      <frameworkLogging channelPriority="WARN">

         <handler type="StreamHandler">
            <param name="msgFormat"><![CDATA[%(asctime)s %(name)s %(message)s]]>
            </param>
         </handler>

      </frameworkLogging>
      <reportLogging channelPriority="WARN">

         <handler type="FileHandler">
            <param name="msgFormat" eval="0"><![CDATA[%(message)s]]></param>
            <param name="fileName" eval="0">puffinResults.log</param>

            <param name="mode" eval="0"><![CDATA[a+]]></param>

         </handler>
      </reportLogging>
      <failureAlertLogging channelPriority="WARN">
         <handler type="FileHandler">

            <param name="msgFormat" eval="0"><![CDATA[%(message)s]]></param>

            <param name="fileName" eval="0">puffinFailures.log</param>
            <param name="mode" eval="0"><![CDATA[a+]]>
            </param>

         </handler>
      </failureAlertLogging>

      <resultsProcessor reportDetail="ALL"/>
      <defaultResponseAnalyzerList>
         <responseAnalyzer src="puffin" type="TimerResponseAnalyzer">
            <param name="timeLimitSecs">10</param>

         </responseAnalyzer>

         <responseAnalyzer src="puffin" type="StatusResponseAnalyzer">
            <param name="httpStatus">200</param>
            <param name="httpStatus">302</param>

         </responseAnalyzer>
      </defaultResponseAnalyzerList>

   </system>
   <testActions/>
</puffin>