I recently worked with a team of software developers who were quite confident with the level of testing they had in place on their project. They had a robust set of unit tests, and extensive code coverage reports to match. They were thoughtful about separating out their unit tests (performed in isolation with the tactical use of mocks and stubs) from their more involved, longer running integration tests.
The project was a traditional website, including a login screen, a catalog of items, and the ubiquitous shopping cart. When I asked them how they tested their user interface, their responses ranged from sheepish to indignant:
Brittleness. Redundancy. A "testing is for finding bugs"-only mindset. All of these seem to conspire against user interface testing, even among developers who consider themselves test-first advocates. Even the phrase "user interface testing" unwittingly seems to place the emphasis on "interface" rather than "user" in the mind of many developers.
Some of this might be explained by the code-first myopia that many software developers suffer from. Inexplicably, "users" are often reduced to a distant, abstract notion rather than being recognized as the whole point of the software writing exercise in the first place.
Which is why I've become a big fan of user journey testing -- from writing the tests to the words we use to describe the tests.
Because, as you'll see in just a moment, there were a number of bugs lurking in their user interface -- bugs that were trivial and obvious to uncover when they simply tested the application by using it as their users would. After months of successfully passing unit tests, the team was surprised by how many undiscovered bugs they still had when they applied a new avenue of testing.
The team of developers were already savvy unit testers, so they were excited and intrigued to tackle a new kind of testing that played to their strengths while gently broadening their perspective.
Here's what a simple "Buy the ingredients for a spaghetti dinner" user journey test looks like in Taiko:
When the developers tried to log into their website in a user journey test, they discovered that they couldn't click the Username and Password fields because they hadn't used the semantically appropriate <label> elements with their <input> fields. They discovered an accessibility bug with the first test they wrote!
They went on to discover that they couldn't programmatically select items from the catalog because, due to a copy/paste error, the items all had the same HTML ID. This functionality had already passed numerous unit tests, but none of the unit tests caught this particular bug because they were testing the shopping cart in isolation instead of testing it through the user interface as their users would use it.
They were caught by surprise when a page that they tried to visit using Taiko errored out because they already had several successfully passing tests in place for the page. The problem was that the existing tests visited the page in isolation and in a simulated environment. When they visited that same page as the user would -- as one step in a sequence of many along the path of a specific user journey -- the test uncovered a tricky state-related bug that they never could've found by testing the page in isolation.
At the end of my engagement with the project, one of the team members said:
"We always knew that testing was important. What we didn't realize is that you unconsciously test what's important to you.
As software developers, we were deeply invested in the quality of our code by writing unit tests. But we failed to dedicate that same level of care to the user experience.
Now that we know more about user journey testing -- and how easy it is! -- we know that our website's user experience will be as rich and bug-free as the developer experience.”
The project was a traditional website, including a login screen, a catalog of items, and the ubiquitous shopping cart. When I asked them how they tested their user interface, their responses ranged from sheepish to indignant:
"Well, we have a couple of Selenium tests, but they're pretty brittle. They always seem to be broken, so we rarely run them."
"We decided that since we test the back end so extensively with unit and integration tests, the UI tests were redundant and unnecessary."
"Since we use [a popular web framework], we assume that the framework is well-tested. What bugs could we possibly find that they haven't found already?"
"I click through every screen of the website right before we give our showcase for the client, just to be sure that nothing blows up in our face."
"We decided that since we test the back end so extensively with unit and integration tests, the UI tests were redundant and unnecessary."
"Since we use [a popular web framework], we assume that the framework is well-tested. What bugs could we possibly find that they haven't found already?"
"I click through every screen of the website right before we give our showcase for the client, just to be sure that nothing blows up in our face."
Brittleness. Redundancy. A "testing is for finding bugs"-only mindset. All of these seem to conspire against user interface testing, even among developers who consider themselves test-first advocates. Even the phrase "user interface testing" unwittingly seems to place the emphasis on "interface" rather than "user" in the mind of many developers.
Some of this might be explained by the code-first myopia that many software developers suffer from. Inexplicably, "users" are often reduced to a distant, abstract notion rather than being recognized as the whole point of the software writing exercise in the first place.
Which is why I've become a big fan of user journey testing -- from writing the tests to the words we use to describe the tests.
Because, as you'll see in just a moment, there were a number of bugs lurking in their user interface -- bugs that were trivial and obvious to uncover when they simply tested the application by using it as their users would. After months of successfully passing unit tests, the team was surprised by how many undiscovered bugs they still had when they applied a new avenue of testing.
What is a User Journey Test?
A user journey is simply the specific steps a user has to perform to accomplish something on a your website. If a user needs to buy ingredients for a spaghetti dinner from a grocery store website, the steps might be:- Visit the website
- Add items (like tomatoes, garlic, Parmesan cheese, and noodles) to the shopping cart
- Click on the Checkout button
The team of developers were already savvy unit testers, so they were excited and intrigued to tackle a new kind of testing that played to their strengths while gently broadening their perspective.
User Journey Testing in Action
The team decided to try out a new browser automation library -- one that is specifically built for user journey testing -- called Taiko.Here's what a simple "Buy the ingredients for a spaghetti dinner" user journey test looks like in Taiko:
openBrowser()
goto('https://thirstyhead.com/groceryworks/')
click('Produce')
click('Vine-Ripened Tomatoes')
click('Purchase')
When the developers tried to log into their website in a user journey test, they discovered that they couldn't click the Username and Password fields because they hadn't used the semantically appropriate <label> elements with their <input> fields. They discovered an accessibility bug with the first test they wrote!
They went on to discover that they couldn't programmatically select items from the catalog because, due to a copy/paste error, the items all had the same HTML ID. This functionality had already passed numerous unit tests, but none of the unit tests caught this particular bug because they were testing the shopping cart in isolation instead of testing it through the user interface as their users would use it.
They were caught by surprise when a page that they tried to visit using Taiko errored out because they already had several successfully passing tests in place for the page. The problem was that the existing tests visited the page in isolation and in a simulated environment. When they visited that same page as the user would -- as one step in a sequence of many along the path of a specific user journey -- the test uncovered a tricky state-related bug that they never could've found by testing the page in isolation.
At the end of my engagement with the project, one of the team members said:
"We always knew that testing was important. What we didn't realize is that you unconsciously test what's important to you.
As software developers, we were deeply invested in the quality of our code by writing unit tests. But we failed to dedicate that same level of care to the user experience.
Now that we know more about user journey testing -- and how easy it is! -- we know that our website's user experience will be as rich and bug-free as the developer experience.”
Technology Hub
An in-depth exploration of enterprise technology and engineering excellence.