Inspection and testing

This section begins with an interesting but somewhat inaccessible article: Tom Gilb's "Planning to Get the Most Out of Inspection." He assumes that the reader has already read his book (I had not) in discussing document inspection as a way to prevent defects in a product. In other words, by discovering defects in project documents -- requirements specifications, design documents, and so forth -- you can prevent people from implementing these problems in the software. The next article, "A Testing Maturity Model for Software Test Process Assessment and Improvement," is on my short list of favorites because it adds so much to our collective body of practical engineering knowledge. Most software engineers are familiar with the Software Capability Maturity Model (CMM), but quality specialists recognize that this model does not address some of the processes and practices specific to testing. A complement to the CMM called the Testing Maturity Model (TMM) is proposed here by Ilene Burnstein, Ariya Homyen, Taratip Suwanassart, Gary Saxena, and Rob Grom. The TMM defines testing-related maturity goals, activities, tasks, and responsibilities that correspond to each of the CMM's five levels. Burnstein et al. also propose a TMM Assessment Model to measure the TMM level of one's own project. The final article in this section, "Choosing a Tool to Automate Software Testing" by Mark Fewster and Dorothy Graham, looks at ways to decide what software to buy. Anyone charged with acquiring a testing tool will find this a very good resource for learning what questions to ask, what to test and look for in software tools, how to predict how well a tool will work in-house, and how to work with vendors, get the most out of demos and trials, and make a final decision.

Audits

In this section, Daughtrey points out that, although many books cover the subject of auditing, most of them are not specific to software audits. "Quality Evaluation of Software Products" by Jørgen Bøegh picks up where the previous article left off: instead of evaluating a tool for purchase, Bøegh's task is to systematically evaluate the quality of a product to determine whether or not it fulfills specified quality requirements. As Bøegh points out, software is increasingly used for such life-critical systems as traffic control, robotics, and medical systems, in which software defects can have extreme consequences. He lists the relevant ISO and IEC standards for software product evaluation and testing, goes on to present several methods for objectively evaluating software quality, and concludes with a real-world example of evaluating software for a fire alarm system. The second article on auditing, "Practical Quality Assurance for Embedded Software" by Erik P .W. M. van Veenendaal, is a case study of how inspections successfully detected and prevented defects in software for a television. By supplementing his company's ISO-9001 procedures with nine person-weeks of inspection time, his team found more than 1,400 defects and was then able to ship a high-quality product. The chapter includes detailed, clear instructions on how to conduct and apply the results of such inspections.

Configuration management

This final section of the book begins with "Software Configuration Management for Project Leaders" by Tim Kasse and Patricia A. McQuaid. They take a broad view of configuration management (CM), defining it not only as source code version control but also as a methodology for controlling change in all software project artifacts -- from design specifications and data files to test procedures and user documentation. CM, they contend, is a critically important process for producing and delivering software in a controlled manner, and they provide many specific details about CM implementation. The final article in the book, "Applying Quantitative Methods to Software Maintenance" by Ed Weller, explains how to use measurement results to make good decisions, and also holds up some commonly-held beliefs for quantitative scrutiny. On one three-year project, for example, the engineers on his team believed that if they were to push bug fixes out faster, they would introduce more new bugs in the process. But when Weller compared the recidivism rate (percent of new bugs) with time spent on fixing bugs, he found no correlation. Throughout the project he improved his group's processes by conducting similar quantitative analyses, and this chapter presents some good lessons for both project managers and process engineers.

Daughtrey ends the book with a reminder that there are triumphs as well as problems in software development projects: Everyone hears about a few well-publicized failures, but many successes go unnoticed. For those of us who spend our days looking for hidden problems in every apparently-functioning bit of software, this is a good reminder. The goal of all our efforts, after all, is to produce something that works.

On the whole, this book presented many useful ideas and techniques; rarely did I find myself muttering "No way, not in my project!" as I read. The collection of articles also accurately reflects the state of the art of software quality engineering. Note that I am not using state-of-the-art in the colloquial sense, as a descriptor for bleeding-edge innovation, but rather to refer to the true state of our current practices, warts and all. Software quality engineering is a discipline that is still in the process of maturing. And I recommend this book for anyone who wants to see a compact but detailed snapshot of what is going on in that particular corner of the software development world.