Master
ThoughtWorks
Menu
Close
  • What we do
    • Go to overview
    • Customer Experience, Product and Design
    • Data Strategy, Engineering and Analytics
    • Digital Transformation and Operations
    • Enterprise Modernization, Platforms and Cloud
  • Who we work with
    • Go to overview
    • Automotive
    • Healthcare
    • Public Sector
    • Cleantech, Energy and Utilities
    • Media and Publishing
    • Retail and E-commerce
    • Financial Services and Insurance
    • Not-for-profit
    • Travel and Transport
  • Insights
    • Go to overview
    • Featured

      • Technology

        An in-depth exploration of enterprise technology and engineering excellence

      • Business

        Keep up to date with the latest business and industry insights for digital leaders

      • Culture

        The place for career-building content and tips, and our view on social justice and inclusivity

    • Digital Publications and Tools

      • Technology Radar

        An opinionated guide to technology frontiers

      • Perspectives

        A publication for digital leaders

      • Digital Fluency Model

        A model for prioritizing the digital capabilities needed to navigate uncertainty

      • Decoder

        The business execs' A-Z guide to technology

    • All Insights

      • Articles

        Expert insights to help your business grow

      • Blogs

        Personal perspectives from ThoughtWorkers around the globe

      • Books

        Explore our extensive library

      • Podcasts

        Captivating conversations on the latest in business and tech

  • Careers
    • Go to overview
    • Application process

      What to expect as you interview with us

    • Grads and career changers

      Start your tech career on the right foot

    • Search jobs

      Find open positions in your region

    • Stay connected

      Sign up for our monthly newsletter

  • About
    • Go to overview
    • Our Purpose
    • Awards and Recognition
    • Diversity and Inclusion
    • Our Leaders
    • Partnerships
    • News
    • Conferences and Events
  • Contact
Global | English
  • United States United States
    English
  • China China
    中文 | English
  • India India
    English
  • Canada Canada
    English
  • Singapore Singapore
    English
  • United Kingdom United Kingdom
    English
  • Australia Australia
    English
  • Germany Germany
    English | Deutsch
  • Brazil Brazil
    English | Português
  • Spain Spain
    English | Español
  • Global Global
    English
Blogs
Select a topic
View all topicsClose
Technology 
Agile Project Management Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionary Architecture Experience Design IoT Languages, Tools & Frameworks Legacy Modernization Machine Learning & Artificial Intelligence Microservices Platforms Security Software Testing Technology Strategy 
Business 
Financial Services Global Health Innovation Retail  Transformation 
Careers 
Career Hacks Diversity & Inclusion Social Change 
Blogs

Topics

Choose a topic
  • Technology
    Technology
  • Technology Overview
  • Agile Project Management
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionary Architecture
  • Experience Design
  • IoT
  • Languages, Tools & Frameworks
  • Legacy Modernization
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Platforms
  • Security
  • Software Testing
  • Technology Strategy
  • Business
    Business
  • Business Overview
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Careers
    Careers
  • Careers Overview
  • Career Hacks
  • Diversity & Inclusion
  • Social Change
Software TestingTechnology

Test Assertions - How do they work?

Twist Team Twist Team

Published: Oct 19, 2010

An introduction to effective test assertions

In the software industry, before we deliver a product, we need to run many test assertions to make sure the product has met the customer's expectation. So what is a test assertion?

What is a test assertion?

In software industry, before we deliver a product, we need to run many test assertions to make sure the productmet customer's expectation.

So what's a test assertion? We can find from internet that, one of the popular definitions is "a test is a condition that must be tested to confirm conformance to a requirement". I quite like thisas it explains quite well what a test assertion is – a condition. And from functional test point of view,the conformance to a piece of requirement often needs more than one assertion.

Due to this, QAs need to work with lots of assertion in their everyday work, either manually or automatically,both depending on the complexity and time. Although no matter whether we run those assertions manually or, there is one thing that we cannot escape, which is designing them in advance, according to our. Good assertion is necessary; otherwise we risk gathering wrong/inaccurate information from QA work.can delay the delivery of software or leave issues uncovered in the product. Since designing good assertionso vital, how to design a good assertion?

Many might say, that's quite easy. Ok, before we come to a conclusion of whether it is easy or not, let'sa look at a very simple example using ThoughtWorks website.

Here we have an acceptance criteria (we use AC for short) as following: Given users on ThoughtWorks homepageusers click on 'Technology' link under 'How we work' drop-down menu Then users navigate to 'Technology'

Let's first take a look at what the homepage looks like:

And what technology page looks like:

Test the acceptance criteria

To test this AC, we need to click on the link ‘Technology’ within drop-down menu ‘How we work’, which is‘Given’(aka pre-condition) of the AC; then make sure we are navigated to technology page, which is‘When’ in AC.

If we try to do that with some code, it can look something like below:

[Action] browser.open("https://www.thoughtworks.com/");

browser.click("/html/body/div[@id='container']/div[@id='header']/div[@id='primary-navigation']/div[@id='block-1']/ul/li[5]/ul/li[1]/a");

browser.waitForPageToLoad("5000");

Once the action is done, we can verify the expected action result ‘Then’ of the AC.

There can be several ways to do the assertions for this simple AC, we show some of them here:

[Assertions]

Way I: We search if the page we navigate to contains 'Technology' and 'How we work'

assertTrue(

  browser.isTextPresent('Technology')

); assertTrue(

  browser.isTextPresent('How we work')

);

Way II: We check if we are navigated to correct URL

assertEquals( "https://www.thoughtworks.com/technology", browser.getLocation() );

Way III: We check if the title of the current page we are navigated to is calledTechnology - www.thoughtworks.com'

assertEquals(

  "Technology - www.thoughtworks.com", browser.getTitle()

);

Way IV: We check if the left hand sidebar has technology highlighted:

assertEquals( browser.getAttribute("//div[@id='container']/div[3]/div[2]/a[1]@class"), "Active");

Re-visit the assertions

What do you think of the above assertions? Right, none of them are good enough for this very simple AC!

Let's take a closer look at them one by one and see what's wrong with them.

Way I:

assertTrue(

  browser.isTextPresent("Technology")

); assertTrue(

  browser.isTextPresent("How we work")

);

It checks if text 'Technology' and 'How we work' present on the page we are on. But they both exist on more thanpage, such as website footer, navigation bar, side bar which are present not only on Technology page. Usingway of doing assertion will hardly see any test fails. As a consequence, we risk passing software to clientfalse confidence.

Way II:

assertEquals( "https://www.thoughtworks.com/technology", browser.getLocation() );

This group checks if we are navigated to right location. If the URL is correct, then it means we are directed tocorrect page. However, this is not enough as the page content itself might be incorrect, there can bebetween the page location and content itself. So test can pass, but the functionality is broken.

Way III:

assertEquals(

  "Technology - www.thoughtworks.com", browser.getTitle()

);

It's good to use different elements on the page for assertion, though it also has the same issue with Way II,test or not won't be able to provide us with accurate information about the status of theunder test.

Way IV:

assertEquals( browser.getAttribute("//div[@id='container']/div[3]/div[2]/a[1]@class"), "Active");

Writing correct assertions

Believe it or not, this one is quite a common mistake we can see in daily testing -- we are testing more than whatneed to. There is an extra business logic tested along with our target functionality; that is why we seeTechnology' option on sidebar highlighted when we are on Technology page. We have a failed test due to this function not working fine while our target functionality is running perfectly.

To do assertion right for this simple case, we have to remember the important principle of writing good assertions - The goal when testing an implementation is to achieve consistent reproducible results that accurately indicate the conformance of the implementation to the specification. In another word: the test assertion will fail if and only if the tested function is broken.

To achieve the goal, we have to examine the part of the software to test carefully, find out what's unique about it, and what we need to find out as a sufficient proof of its working.

Now let's come back to the Technology page and see what we can use.

First, we can find that the URL is supposed to be used only by 'Technology' page.

Second, the title of the page should also be correct while we are on the 'Technology' page.

But we have not yet proven that the page content is correct. So we have to find something unique in the page content. Examining carefully we can see that the purple span is something unique. It could be in another color which can be described on another AC. So we don't need that in this case, but have a span named Technology which should be unique.

Once we get these three identifiers, we can start to write our automated test again if we want.

It could be code as following:

assertEquals( "https://www.thoughtworks.com/technology", browser.getLocation() );

assertEquals( "Technology - www.thoughtworks.com", browser.getTitle());

assertEquals(browser.getText("/html/body/div[@id='container']/div[3]/p[1]/span"), 'Technology');

Now our test fails only when our functionality is not working. We can confidently say when this test is passed; that this piece of software is ready to be delivered.

So to be able to write good assertion, we need to understand the underlying checkpoints of a piece of requirement, make sure we go from the correct entry to the point we need to test, we need to find out unique identifiers to avoid confusion of the test result. Also, we need to be aware of not including implicit business logic that is not involved in the requirement to test, and, we have to understand that to do good assertion of a piece of requirement, we might need more than one assertion.

To sum up, analysis is just too important for writing good assertion.

Master
Privacy policy | Modern Slavery statement | Accessibility
Connect with us
×

WeChat

QR code to ThoughtWorks China WeChat subscription account
© 2021 ThoughtWorks, Inc.