BDD is limited in what it can do for the team

Behavioral-Driven Development (BDD) requires the team to

  1. develop a “ubiquitous language” that describes the SUT in terms that anybody in the business can understand
  2. Create a custom language (e.g. Gherkin) with the “given-then-when” syntax that describes what the product needs to do, in place of functional requirements
  3. Use a custom parser (e.g. Cucumber) and deploy that parser to execute the script at any tier that supports the automation
  4. Implement keywords (or, key phrases) that actually drive and measure the SUT.

There might be BDD experts out there that know what BDD is better than I do; if any of you think I got it wrong, comment and let’s work it the details.

Item 1 is the biggest promise of BDD: collaboration across the business. That’s valuable, if it works better than what came before. From what I hear, though, having the PO’s and PM’s write automation routines hasn’t really worked out, and the QA role ends up doing that anyway.

Items 2 and 3 seem like they could create extra quality risk. Choosing and implementing keywords or phrases is not always obvious or unambiguous.

It’s item 4 that prevents BDD from really delivering quality value to the degree that we need: implementations of keywords or phrases are obscure (I know, I’ve written and implemented them) to anybody without fingers in the automation code. Log statements help a bit, but logs are limited in what they can illuminate because they are a tool from a very different problem domain; for example see my earlier post here.

The keywords or phrases are highlighted as “self-documenting” by people who teach BDD, but they’re not:

First, it is the script interpreter that documents the keywords, not the keywords themselves.

Second, what happens behind the keyword is in that hidden implementation. If you are willing to look at the code, you might get an idea, but to really know, you must step through with a debugger – assuming that you can reproduce whatever SUT state you are interested in.

If you want automation that documents itself in trustworthy detail at runtime, and in a format that is both approachable by people (in a web browser or with a stylesheet) and automation-friendly, you need to rethink how you code your automation. Samples here show one way to do it.

With MetaAutomation, the self-documenting nature of the automation is right there in the code; no parser or made-up language needed. The documentation created at runtime comes directly from SUT behavior and the behavior of automation code, dependencies and harnesses. Each step in the hierarchy has a hard-coded, readable name. Procedure changes and variations, and error-handling, are more manageable in a hierarchy, because it’s a natural pattern – see here for example.

Roles that do not care about the technical details of driving the SUT can look at the root or top-level business-facing levels of the hierarchy. Drilling down to the technology-facing atomic steps of the check is an option for anybody on the team, but of special importance to QA role and developers.

BDD is a clever idea, and it goes in the correct direction: towards more transparency and better communication around the team. Unfortunately it only goes about 10% of the distance to where we need to be to ship software faster and at higher quality.

No Comments

Add a Comment

Sign up here for emails with bites of wisdom on quality automation and MetaAutomation

Recent Blog Posts

  • The differences: Manual Test vs. quality automation

    In my last post I describe out the two kinds of automation that fit in the quality automation space.

    People who do quality automation (at least, the part of quality automation that drives and … more

  • The two halves of quality automation

    Quality automation is the domain (or problem space) of driving the SUT, measuring and recording data on SUT behavior and communicating that data to the business. I also use “quality automation” to … more

  • Fixing the false negative problem

    False negatives happen when these three things happen in order:

    Operations (ops) promotes the software to the next level, or ships it to end-users

    Someone (or, some automated process) discovers a … more

  • Fixing the false positive problem

    With all the quality automation that is your responsibility, a run of a check failed. It is your job to check it out.

    After 30 minutes or so of investigation, you find that the failure happened … more