Keep your eyes on the DSL prize with Twist

Learning a language can be a challenging task.  The absorption of a lexicon takes time and patience.  Twist can help keep this task achievable for consumers of acceptance tests by allowing the definition of confirmation language to be natural.  This is assuming that the consumers will be business customers or other non-technical people.  If that is not the case, I would suggest using a pure code test framework such as TestNG to achieve the technical benefits of the Twist runner design.

Assuming the later statement is the case, Twist has the potential to facilitate communication in and around software delivery teams on a dramatic scale.  It can help give teams a highly accesible language to express acceptance tests that give common understanding, and enable rapid continuous delivery through automation.  However, it does take discipline to keep that goal in mind, and not to get side tracked on the technicalities of coding the automation solution.  It’s not that those things don’t need consideration.  On the contrary, it’s good to be thinking about the execution giving the fastest possible feedback, producing maintainable automation code, and ideally even test data abstraction.  The fact is that those things won’t matter if the language for communication is neglected as the focus.

Content Assist

Twist has an excellent set of features for managing a scenario focused language.  In particular, it allows the following:

• Creation of method signatures and code from workflow steps (in natural language)
• Rephrasing of workflow steps across scenarios and the use of searching by content assist
• Abstraction of multiple workflow steps in to concepts or code implementation (the first is preferred for language clarity)
• Data driving of workflow step parameters
• … and an ever growing list

Limit The Deviation and Limit The Confusion

The experience I’ve had is that it is easy to create a massive amount of confusion (and also code) if you don’t keep to some basic patterns of usage around the language used and some ground rules.

Usage Patterns

• Break down workflow fixtures by pages / areas / services to give them context
• Before creating a new workflow step search for an existing match
• When defining a new workflow step ensure it can be easily searched for by others

Ground Rules

• To support the searching of workflow steps allow the context of the area of the system under test to be visible
  • e.g. “… on the home page” or “… using google search”
• Keep to a basic set of action words against each area so that it can display the interactions it allows easily
• e.g. “Verify search suggestions for ‘Simon Reekie’ using google search”
• e.g. “Set language to ‘English’ on the home page”
• e.g. “Set currency to ‘British Pounds’ on the home page”
• e.g. “Get and Store breakfast price as “Reported Breakfast Price” on the price page
• e.g. “Verify breakfast price is equal to Stored “Report Breakfast Price” on the summary page
• If you are going to get something off one part of the system under test and verify it in another, then keep it visible within the scenario language

Conclusion

It is difficult enough communicating with many individuals to a common understanding, and keeping up with managing the complexity of that communication.  Give yourself a chance by building a DSL for Twist scenarios that keeps to a few understood rules, and works within the boundaries of the tool capabilities.  There is nothing wrong with adding or changing the rules, but do it knowingly and collaboratively with those using them.

For those that are looking for the technical implications of this approach, I can only really give you these figures as food for thought.

• Before this approach was taken in my current workplace, and scenario workflow step language spaghetti was everywhere… we had:
  • Thousands of fixture and test implementation classes built up with duplication (I know… refactoring mindsets could have helped)
  • It took days in some cases for all types of people to write a useful automated scenario
  • The debugging of test failures out of CI took an hours and became a specialist skill
• After this approach was embedded we had the same coverage plus much more, but we also had:
• A couple of dozen fixture classes
• A whole range to skill levels could build new scenarios to point of automation within minutes to an hour

Guest Blog by Simon Reekie resposted from http://blog.simon-reekie.me/2011/11/05/twist-dsl-prize/