The “Four-Phase Test” Pattern is obsolete

In 2007, Gerard Meszaros published his tome “xUnit Test Patterns.” It is an impressive, well-done, scholarly work of cartography. Gerard delivered to the software quality community a definitive work of naming and describing commonly used patterns. Now we now how to describe and how to communicate many patterns of applying automation to software quality. Wonderful! Now we know how to bring order to the chaotic madness of what is commonly called “test automation…” right?

I owe Gerard a debt of gratitude, so I put a paragraph about him in my “Gratitude” section in the MetaAutomation 3rd edition. He suggested the name “Precondition Pool” for one of the patterns of MetaAutomation, and I used it because it works at least as well as anything I could come up with.

I do not expect anything of anybody else that I would not expect of myself; I therefore write with the utmost sincerity and anticipated reciprocity that I thank Gerard and note here that Four-Phase Test (page 358, xUnit Test Patterns, Meszaros 2007) is obsolete.

What, the setup-procedure-verification-cleanup pattern is obsolete? Yes, the internet and connectivity, and the objective of shipping software faster at higher quality, obsoletes it.

To ship software faster and at higher quality, we must have our automation record (i. e., pay attention) to how it drives and measures the SUT, and how the SUT responds. The information must be recorded in an automation-friendly format.

The highly-interconnected aspects of applications that matter to people mean that there is no clean distinction between the four phases of Four-Phase Test, and it means that they are more equally important. Even if they were cleanly distinct – which I submit they are not - all four phases have important expected behaviors.

So, treat them as one, or treat them as distinct but equally important aspects of testing the SUT (as with the Precondition Pool pattern working with the Atomic Check pattern of MetaAutomation).

Having automation pay attention to how it drives the SUT means that it makes no sense to throw information away with an “abort” status from a setup failure — it’s all important to monitoring software quality and delivering metrics on the SUT. This is more important than a mere “input” to the test — it is integral to the quality aspect we are testing. This is all important data for automated and human follow-up and analysis on SUT quality.

Too, placing the “setup” phase inline with the rest of the automated check can interfere with scale. If the “setup” phase really is independent of the rest of the check in the sense that it could be done out-of-process, then it should be handled by the Precondition Pool pattern (see the pattern map here), including error handling, cleanup, and human intervention as needed, *or* as part of the check, where the artifact of the check is a single ordered tree. (link here) This works best for both human and automated follow-up.

IMO “Four-Phase Test” has helped, and it’s time to move on to the next paradigm so we can ship software faster and at higher quality.

No Comments

Add a Comment

Sign up here for emails with bites of wisdom on quality automation and MetaAutomation

Recent Blog Posts

  • The differences: Manual Test vs. quality automation

    In my last post I describe out the two kinds of automation that fit in the quality automation space.

    People who do quality automation (at least, the part of quality automation that drives and … more

  • The two halves of quality automation

    Quality automation is the domain (or problem space) of driving the SUT, measuring and recording data on SUT behavior and communicating that data to the business. I also use “quality automation” to … more

  • Fixing the false negative problem

    False negatives happen when these three things happen in order:

    Operations (ops) promotes the software to the next level, or ships it to end-users

    Someone (or, some automated process) discovers a … more

  • Fixing the false positive problem

    With all the quality automation that is your responsibility, a run of a check failed. It is your job to check it out.

    After 30 minutes or so of investigation, you find that the failure happened … more