Make your automation sit up and pay attention

One of the problems with the phrase “test automation” is that it implies that what a manual tester can do can also be automated, and even that it is automated when someone automates a manual test case.

By standard practices, though, an automated procedure does not have the capability of paying attention; it loses all that information for how it drove and measured the SUT. This is automation for quality measurement and management, or as I call it, quality automation. The fact that the automation pays no attention to what it is doing is equivalent to pouring business value on the floor. Using Gherkin for keyword-based automation makes a small step in the right direction of making automation visible in business language, but the keyword implementations are not visible so the trustworthiness of the result of a check run becomes suspect (for good reason; I’ve done this).

Log statements can help, but they are an ideal tool from a different problem domain, so they really aren’t up to it; logs drop all context (other than timestamp) by default.

There is a much more effective way to preserve that business value, and although it can’t replace the manual testing role, it does use a pattern that we all use, every day: Hierarchical Steps. (see here for more on Hierarchical Steps and some nice diagrams.)

GitHub-hosted samples here show Hierarchical Steps implemented in Atomic Check, in C#. These solutions use the free version of Visual Studio, but other than the services projects, they are platform-independent.

Implementing Hierarchical Steps to preserve the what the automation is doing supports some very powerful stuff; for example, now any role can access information on performance of every step and what the SUT is doing exactly, by drilling down into as much detail as (s)he wishes.

The rest of the MetaAutomation pattern language here shows other awesome stuff you can do with good information from your automation.

There’s great business value available from having your automation sit up and pay attention.

No Comments

Add a Comment

Sign up here for emails with bites of wisdom on quality automation and MetaAutomation

Recent Blog Posts

  • A gift to company executives

    For automation for quality (a superset of/fix for what is often called “test automation”) MetaAutomation provides trustworthy detail and transparency on what the SUT is doing and how fast to all … more

  • Ruby has lost its sparkle

    Ruby is very popular as an automation tool.

    Ruby’s creator, Yukihiro Matsumoto, had what seemed like great ideas: programmer productivity and fun, and that (to quote Wikipedia, quoting Matsumoto)

    “ … more

  • Make your automation sit up and pay attention

    One of the problems with the phrase “test automation” is that it implies that what a manual tester can do can also be automated, and even that it is automated when someone automates a manual test … more

  • The limits of AI testing

    Artificial intelligence (AI, or more accurately called machine learning) is all over public discussions about software testing. It seems like the next big thing! Many imply that it will take over the … more