Introduction to Puffin

As important as unit testing is, it fails to test the overall functionality of an application as a "system." Just because each functional component works as designed through the construction of its unit tests, that doesn't mean that those components -- when brought together into a coherent application -- will work together to meet the system's overall functional requirements. System-wide functional testing is an important part of deployment of all applications; and Web applications are no exception. However, the tools to enable full system testing are either cost prohibitive or overly complex, often to the point that system-wide testing is done only cursorily or a human being is forced to test each functional area manually.

Human testing can never (and arguably should never) be completely removed from the testing process. However, it can be argued that improving automation of testing can free the quality engineer to focus on the most complex functionality.

Complex testing aside, there is another benefit to creating automated smoke tests of your entire Web application. Most systems -- especially large systems -- are in a constant state of change in response to evolving business needs. Just as unit testing allows you to make pervasive (even "extreme") changes to a given unit with confidence that those changes will not affect that unit's behaviour, a smoke test of the entire system would allow you that same confidence across the entire system.

Finally, system-wide smoke tests -- specifically for Web applications -- solve a problem of many unit testing frameworks: changes in technology. Unit tests -- even a long series of unit tests -- are very technology specific, forcing you to rewrite every component (and its corresponding unit test) at the same time or testing only some portion of the units making up the system. For example, suppose you are converting a CGI application to a JSP/servlet architecture. In this scenario there are doubtless some components that can be converted separately from the others. Only with a system-wide smoke test will you be able to test both the older CGI components and the newer JSP/servlet components and their integration into a whole system during the migration process.

This last capability points out a shortcoming in many system-testing tools: technology specificity. There are plenty of system-testing tools out there. Some are even open source. However, most are technology specific, making changes to the underlying technology difficult. The result? Most people either write unmaintainable scripts to test the back-and-forth HTTP traffic from the Web application, or worse, rely 100% on human testing.

What's needed, then, is a Web application-testing tool that allows for:

• End-to-end system testing with dynamically created test actions based on results from previous test actions.
• Using the same test to test the same functionality -- regardless of the Web application's underlying technology. You should be able to test the Web application as if it were a proverbial "black box."

Furthermore, the following would be "nice to have" in a Web application-testing tool:

• Ease of use (less need for scripting, etc.)
• Extensibility
• Flexibility, flexibility, flexibility...

This article introduces Puffin, a Web application-testing framework that provides developers of even the most complex Web applications all of the above features. This introduction covers how to get and install Puffin, how to configure the system to your Web application's needs, and finally how to write simple test plans.