What I dislike about BDD / Cucumber: Architectural issues

Cucumber relies on statements about SUT behavior organized in given/when/then syntax, also called Gherkin language. This forces authors to break their statements up and organize them by preconditions, operations, and verification(s). The syntax seems like a reasonable compromise between English and Computer code. So far, so good…

On pages 41-42 the authors describe how these “keywords” as they call them – given, when, then, etc. – are actually ignored at runtime. They are there for documentation only! Hence the “tiger trap” I describe on this post.

Is there any other language that doesn’t always clearly discriminate between executable code and comments? I do not know of one. There is a reason for that; if coders do not know what text effects what the computer does and what doesn’t, it can cause confusion, wasted time and stress.

The authors probably made the choice to make the keywords irrelevant for runtime execution so that they could support keyword highlighting in the language of the user’s choice, as on page 42 in the section “Speaking in Tongues,” or for using a silly and obfuscating choice like pirate or LOLCAT (page 195) (see also this post).

On the topic of obfuscation, on page 195 the authors describe the command “-- guess" to do a runtime cover up of design issues resulting from what the automation authors have created in the expressions that select runtime behavior. The keyword “guess” suggests randomness, but it’s really not random, it’s more like “tolerate ambiguity by making a choice according to an established algorithm.” This can be a desirable attribute in someone who reports to you, because you don’t want that person asking you for unnecessary clarification on what to do next, but in computer code, it just covers up design and architecture issues to create a new kind of ambiguity: what happened with the automation? Oh, we have to dig into the logs to find out, because the Cucumber step methods design is creating ambiguity. It’s a lazy way to get the automation to do something with the SUT that can also hide information on what happened and what were the results. This can create quality risk when it becomes unclear what exactly the automation covers and what should be done manually to try to compensate.

I am guessing that the authors made the choice to support the command “-- guess" out of the common misunderstanding around software quality practices that comes from the mistaken 1979 assertion by Glenford Myers (in his book “The Art of Software Testing”) that “The whole point of test is to find bugs.” I write a lot more about this problem elsewhere. If the purpose of automation were just to find bugs, then knowledge of what the automation does with the SUT, and automation results, becomes much less important than just driving the SUT to do stuff. See also my post here.

The shift-left movement in software quality is about surfacing quality knowledge sooner rather than later. Quality knowledge is more useful, less randomizing, and less risky to the team when it’s available quickly.

By contrast, on pages 124-126 of the book, there is a section “Organizing the Code” with discussion of inter-file dependencies and rules for managing quality risk. There is a lot of quality risk in the Cucumber system due to complexity and compounded with the fact that any errors or ambiguities about Cucumber finding steps are deferred to runtime.

On page 87, the section “Avoiding Incidental Details” is really about burying details so that only the automation authors have access, and only by code inspection or stepping through them. This section speaks to a motivator of my work with quality automation (MetaAutomation) to surface all those details so they are available to any role or any individual that cares about SUT quality.


Previous posts in this series:

Future post (linked, as the post happens):

No Comments

Add a Comment

Sign up here for emails with bites of wisdom on quality automation and MetaAutomation

Recent Blog Posts