Make your automation sit up and pay attention

One of the problems with the phrase “test automation” is that it implies that what a manual tester can do can also be automated, and even that it is automated when someone automates a manual test case.

By standard practices, though, an automated procedure does not have the capability of paying attention; it loses all that information for how it drove and measured the SUT. This is automation for quality measurement and management, or as I call it, quality automation. The fact that the automation pays no attention to what it is doing is equivalent to pouring business value on the floor. Using Gherkin for keyword-based automation makes a small step in the right direction of making automation visible in business language, but the keyword implementations are not visible so the trustworthiness of the result of a check run becomes suspect (for good reason; I’ve done this).

Log statements can help, but they are an ideal tool from a different problem domain, so they really aren’t up to it; logs drop all context (other than timestamp) by default.

There is a much more effective way to preserve that business value, and although it can’t replace the manual testing role, it does use a pattern that we all use, every day: Hierarchical Steps. (see here for more on Hierarchical Steps and some nice diagrams.)

GitHub-hosted samples here show Hierarchical Steps implemented in Atomic Check, in C#. These solutions use the free version of Visual Studio, but other than the services projects, they are platform-independent.

Implementing Hierarchical Steps to preserve the what the automation is doing supports some very powerful stuff; for example, now any role can access information on performance of every step and what the SUT is doing exactly, by drilling down into as much detail as (s)he wishes.

The rest of the MetaAutomation pattern language here shows other awesome stuff you can do with good information from your automation.

There’s great business value available from having your automation sit up and pay attention.

No Comments

Add a Comment

Sign up here for emails with bites of wisdom on quality automation and MetaAutomation

Recent Blog Posts