What I dislike about BDD / Cucumber: Promises, promises…

The book opens (page 3) with a well-written section on communication between stakeholders and programmers. On page 5, “What makes Cucumber stand out from the crowd of other testing tools is that it has been designed specifically to ensure the acceptance tests can easily be read – and written – by anyone on the team.” On page 6 they describe the Gherkin steps as “executable specifications” and “living documentation.”

Sounds great, but they don’t mention some existing barriers to communication inherent in their system:

  • The glue code contains details that can be important to developers and manual testers, for example, which controls are used (there might be multiple choices) for a given operation. These details are hidden in source code that only the authors of that code will ever bother looking at, and to really know what’s going on at runtime, the authors must step through with a debugger.
  • Cucumber selects the implementation steps at runtime by matching strings with regular expressions. The implementations may be organized by file directory. A change to these strings could easily cause unexpected downstream changes in which method is used for the keyword, but the true behavior of the system won’t be clear until runtime.
  • Communication between implemented steps relies on global variables, with all the downsides you can expect from using globals, e.g., a modification in one step unexpectedly changing the global variable for another step.
  • The steps that make up a Cucumber check make a linear list, which severely limits documentation clarity. Cucumber allows nested steps, but they appear very awkward and the book recommends strongly (page 75) that you not use this feature. The check steps are limited to a linear list and this limits clarity and flexibility for reading, executing, reporting, error handling, and editing. (That is why I use the Hierarchical Steps pattern of MetaAutomation, with open-source sample implementations, to solve these problems.)
  • The authors use the script language Ruby, which defers errors to runtime, just as the Cucumber tool does. This is the opposite of “shift left” and it complicates communication and workflow when errors appear long after they are introduced (as has happened to me with Ruby)

Previous posts in this series:

Future posts (with links, as the posts happen):

No Comments

Add a Comment

Sign up here for emails with bites of wisdom on quality automation and MetaAutomation

Recent Blog Posts