Running a test plan

You have a test plan. Now it's time to execute and see what this whole smoke-test thing is all about. Here is all that is required:

1. Make sure the demo application is running and that you can hit it using your browser or other appropriate client of the Web application (remember Puffin isn't just for Web sites -- it can even test Web services).
   
   
2. On the command line, navigate to the folder containing puffin.py and type the following:
   
   puffin

If not specified, Puffin will use the following (enter puffin --help for a full run-down on command-line arguments):

• test.plan=test.plan
• puffin.conf=puffinConfig.xml
• logging=WARN

You should see the following:

   Starting Puffin.
   Initializing framework.
   Beginning test plan execution.

   1. Executing task: login.
   ----Test action: login --> SUCCESS.
   Task result: SUCCESS.

   2. Executing task: getPetsList.
   ----Test action: getPetsList --> SUCCESS.
   Task result: SUCCESS.

   3. Executing task: getPostersList.
   ----Test action: getPostersList --> SUCCESS.
   Task result: SUCCESS.

   4. Executing task: tooLong.
   ----Test action: tooLong --> FAILURE.
   Task result: FAILURE.

   5. Executing task: serverError.
   ----Test action: serverError --> FAILURE.
   Task result: FAILURE.

   Ending test plan execution.
   Ending Puffin.

Note that at #4, Puffin will appear to hang. This is exactly right as this script should take 30 seconds to execute (thus failing in terms of the TimerResponseAnalyzer).

If you got an error, check the following:

1. All Puffin requirements are installed properly on your machine (Python, eGenix Base, PyXML, Puffin itself).
2. Your puffinConfig.xml is both a valid XML document and correct.
3. You don't have any spurious spellings in your test.plan file.

Here's what Puffin did:

1. Loaded the config file (in this case the default) and initialized itself.
2. Loaded the test plan and initialized it with the information from the configuration file.
3. Executed each test action in the order it was listed in the test plan. For each test action, this involves the following sub-steps:
   1. It processed any inputs.
   2. It made the call to the Web application and retrieved its results.
   3. It analyzed the results using both the StatusResponseAnalyzer and the TimerResponseAnalyzer.
   4. It processed any outputs and placed their values into the current token dictionary.
4. Reported the status for each action on the command line (using the StreamHandler, remember).
5. Wrote a file (puffinResults.log) containing a full report of the entire test plan's execution. If this file already exists, Puffin appends the results onto the end of the file.
6. For each failure, Puffin wrote out a failure alert to a file in this case (puffinFailures.log). This could just as easily have been an e-mail address or similar.
7. Puffin closed down.