ThoughtWorks
  • Contact
  • Español
  • Português
  • Deutsch
  • 中文
Go to overview
  • Engineering Culture, Delivery Mindset

    Embrace a modern approach to software development and deliver value faster

    Intelligence-Driven Decision Making

    Leverage your data assets to unlock new sources of value

  • Frictionless Operating Model

    Improve your organization's ability to respond to change

    Platform Strategy

    Create adaptable technology platforms that move with your business strategy

  • Experience Design and Product Capability

    Rapidly design, deliver and evolve exceptional products and experiences

    Partnerships

    Leveraging our network of trusted partners to amplify the outcomes we deliver for our clients

Go to overview
  • Automotive
  • Cleantech, Energy and Utilities
  • Financial Services and Insurance
  • Healthcare
  • Media and Publishing
  • Not-for-profit
  • Public Sector
  • Retail and E-commerce
  • Travel and Transport
Go to overview

Featured

  • Technology

    An in-depth exploration of enterprise technology and engineering excellence

  • Business

    Keep up to date with the latest business and industry insights for digital leaders

  • Culture

    The place for career-building content and tips, and our view on social justice and inclusivity

Digital Publications and Tools

  • Technology Radar

    An opinionated guide to technology frontiers

  • Perspectives

    A publication for digital leaders

  • Digital Fluency Model

    A model for prioritizing the digital capabilities needed to navigate uncertainty

  • Decoder

    The business execs' A-Z guide to technology

All Insights

  • Articles

    Expert insights to help your business grow

  • Blogs

    Personal perspectives from ThoughtWorkers around the globe

  • Books

    Explore our extensive library

  • Podcasts

    Captivating conversations on the latest in business and tech

Go to overview
  • Application process

    What to expect as you interview with us

  • Grads and career changers

    Start your tech career on the right foot

  • Search jobs

    Find open positions in your region

  • Stay connected

    Sign up for our monthly newsletter

Go to overview
  • Conferences and Events
  • Diversity and Inclusion
  • News
  • Open Source
  • Our Leaders
  • Social Change
  • Español
  • Português
  • Deutsch
  • 中文
ThoughtWorksMenu
  • Close   ✕
  • What we do
  • Who we work with
  • Insights
  • Careers
  • About
  • Contact
  • Back
  • Close   ✕
  • Go to overview
  • Engineering Culture, Delivery Mindset

    Embrace a modern approach to software development and deliver value faster

  • Experience Design and Product Capability

    Rapidly design, deliver and evolve exceptional products and experiences

  • Frictionless Operating Model

    Improve your organization's ability to respond to change

  • Intelligence-Driven Decision Making

    Leverage your data assets to unlock new sources of value

  • Partnerships

    Leveraging our network of trusted partners to amplify the outcomes we deliver for our clients

  • Platform Strategy

    Create adaptable technology platforms that move with your business strategy

  • Back
  • Close   ✕
  • Go to overview
  • Automotive
  • Cleantech, Energy and Utilities
  • Financial Services and Insurance
  • Healthcare
  • Media and Publishing
  • Not-for-profit
  • Public Sector
  • Retail and E-commerce
  • Travel and Transport
  • Back
  • Close   ✕
  • Go to overview
  • Featured

  • Technology

    An in-depth exploration of enterprise technology and engineering excellence

  • Business

    Keep up to date with the latest business and industry insights for digital leaders

  • Culture

    The place for career-building content and tips, and our view on social justice and inclusivity

  • Digital Publications and Tools

  • Technology Radar

    An opinionated guide to technology frontiers

  • Perspectives

    A publication for digital leaders

  • Digital Fluency Model

    A model for prioritizing the digital capabilities needed to navigate uncertainty

  • Decoder

    The business execs' A-Z guide to technology

  • All Insights

  • Articles

    Expert insights to help your business grow

  • Blogs

    Personal perspectives from ThoughtWorkers around the globe

  • Books

    Explore our extensive library

  • Podcasts

    Captivating conversations on the latest in business and tech

  • Back
  • Close   ✕
  • Go to overview
  • Application process

    What to expect as you interview with us

  • Grads and career changers

    Start your tech career on the right foot

  • Search jobs

    Find open positions in your region

  • Stay connected

    Sign up for our monthly newsletter

  • Back
  • Close   ✕
  • Go to overview
  • Conferences and Events
  • Diversity and Inclusion
  • News
  • Open Source
  • Our Leaders
  • Social Change
Blogs
Select a topic
View all topicsClose
Technology 
Agile Project Management Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionary Architecture Experience Design IoT Languages, Tools & Frameworks Legacy Modernization Machine Learning & Artificial Intelligence Microservices Platforms Security Software Testing Technology Strategy 
Business 
Financial Services Global Health Innovation Retail  Transformation 
Careers 
Career Hacks Diversity & Inclusion Social Change 
Blogs

Topics

Choose a topic
  • Technology
    Technology
  • Technology Overview
  • Agile Project Management
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionary Architecture
  • Experience Design
  • IoT
  • Languages, Tools & Frameworks
  • Legacy Modernization
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Platforms
  • Security
  • Software Testing
  • Technology Strategy
  • Business
    Business
  • Business Overview
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Careers
    Careers
  • Careers Overview
  • Career Hacks
  • Diversity & Inclusion
  • Social Change
Software TestingTechnology

Test Assertions - How do they work?

Twist Team Twist Team

Published: Oct 19, 2010

An introduction to effective test assertions

In the software industry, before we deliver a product, we need to run many test assertions to make sure the product has met the customer's expectation. So what is a test assertion?

What is a test assertion?

In software industry, before we deliver a product, we need to run many test assertions to make sure the productmet customer's expectation.

So what's a test assertion? We can find from internet that, one of the popular definitions is "a test is a condition that must be tested to confirm conformance to a requirement". I quite like thisas it explains quite well what a test assertion is – a condition. And from functional test point of view,the conformance to a piece of requirement often needs more than one assertion.

Due to this, QAs need to work with lots of assertion in their everyday work, either manually or automatically,both depending on the complexity and time. Although no matter whether we run those assertions manually or, there is one thing that we cannot escape, which is designing them in advance, according to our. Good assertion is necessary; otherwise we risk gathering wrong/inaccurate information from QA work.can delay the delivery of software or leave issues uncovered in the product. Since designing good assertionso vital, how to design a good assertion?

Many might say, that's quite easy. Ok, before we come to a conclusion of whether it is easy or not, let'sa look at a very simple example using ThoughtWorks website.

Here we have an acceptance criteria (we use AC for short) as following: Given users on ThoughtWorks homepageusers click on 'Technology' link under 'How we work' drop-down menu Then users navigate to 'Technology'

Let's first take a look at what the homepage looks like:

And what technology page looks like:

Test the acceptance criteria

To test this AC, we need to click on the link ‘Technology’ within drop-down menu ‘How we work’, which is‘Given’(aka pre-condition) of the AC; then make sure we are navigated to technology page, which is‘When’ in AC.

If we try to do that with some code, it can look something like below:

[Action] browser.open("https://www.thoughtworks.com/");

browser.click("/html/body/div[@id='container']/div[@id='header']/div[@id='primary-navigation']/div[@id='block-1']/ul/li[5]/ul/li[1]/a");

browser.waitForPageToLoad("5000");

Once the action is done, we can verify the expected action result ‘Then’ of the AC.

There can be several ways to do the assertions for this simple AC, we show some of them here:

[Assertions]

Way I: We search if the page we navigate to contains 'Technology' and 'How we work'

assertTrue(

  browser.isTextPresent('Technology')

); assertTrue(

  browser.isTextPresent('How we work')

);

Way II: We check if we are navigated to correct URL

assertEquals( "https://www.thoughtworks.com/technology", browser.getLocation() );

Way III: We check if the title of the current page we are navigated to is calledTechnology - www.thoughtworks.com'

assertEquals(

  "Technology - www.thoughtworks.com", browser.getTitle()

);

Way IV: We check if the left hand sidebar has technology highlighted:

assertEquals( browser.getAttribute("//div[@id='container']/div[3]/div[2]/a[1]@class"), "Active");

Re-visit the assertions

What do you think of the above assertions? Right, none of them are good enough for this very simple AC!

Let's take a closer look at them one by one and see what's wrong with them.

Way I:

assertTrue(

  browser.isTextPresent("Technology")

); assertTrue(

  browser.isTextPresent("How we work")

);

It checks if text 'Technology' and 'How we work' present on the page we are on. But they both exist on more thanpage, such as website footer, navigation bar, side bar which are present not only on Technology page. Usingway of doing assertion will hardly see any test fails. As a consequence, we risk passing software to clientfalse confidence.

Way II:

assertEquals( "https://www.thoughtworks.com/technology", browser.getLocation() );

This group checks if we are navigated to right location. If the URL is correct, then it means we are directed tocorrect page. However, this is not enough as the page content itself might be incorrect, there can bebetween the page location and content itself. So test can pass, but the functionality is broken.

Way III:

assertEquals(

  "Technology - www.thoughtworks.com", browser.getTitle()

);

It's good to use different elements on the page for assertion, though it also has the same issue with Way II,test or not won't be able to provide us with accurate information about the status of theunder test.

Way IV:

assertEquals( browser.getAttribute("//div[@id='container']/div[3]/div[2]/a[1]@class"), "Active");

Writing correct assertions

Believe it or not, this one is quite a common mistake we can see in daily testing -- we are testing more than whatneed to. There is an extra business logic tested along with our target functionality; that is why we seeTechnology' option on sidebar highlighted when we are on Technology page. We have a failed test due to this function not working fine while our target functionality is running perfectly.

To do assertion right for this simple case, we have to remember the important principle of writing good assertions - The goal when testing an implementation is to achieve consistent reproducible results that accurately indicate the conformance of the implementation to the specification. In another word: the test assertion will fail if and only if the tested function is broken.

To achieve the goal, we have to examine the part of the software to test carefully, find out what's unique about it, and what we need to find out as a sufficient proof of its working.

Now let's come back to the Technology page and see what we can use.

First, we can find that the URL is supposed to be used only by 'Technology' page.

Second, the title of the page should also be correct while we are on the 'Technology' page.

But we have not yet proven that the page content is correct. So we have to find something unique in the page content. Examining carefully we can see that the purple span is something unique. It could be in another color which can be described on another AC. So we don't need that in this case, but have a span named Technology which should be unique.

Once we get these three identifiers, we can start to write our automated test again if we want.

It could be code as following:

assertEquals( "https://www.thoughtworks.com/technology", browser.getLocation() );

assertEquals( "Technology - www.thoughtworks.com", browser.getTitle());

assertEquals(browser.getText("/html/body/div[@id='container']/div[3]/p[1]/span"), 'Technology');

Now our test fails only when our functionality is not working. We can confidently say when this test is passed; that this piece of software is ready to be delivered.

So to be able to write good assertion, we need to understand the underlying checkpoints of a piece of requirement, make sure we go from the correct entry to the point we need to test, we need to find out unique identifiers to avoid confusion of the test result. Also, we need to be aware of not including implicit business logic that is not involved in the requirement to test, and, we have to understand that to do good assertion of a piece of requirement, we might need more than one assertion.

To sum up, analysis is just too important for writing good assertion.

  • What we do
  • Who we work with
  • Insights
  • Careers
  • About
  • Contact

WeChat

×
QR code to ThoughtWorks China WeChat subscription account

Media and analyst relations | Privacy policy | Modern Slavery statement ThoughtWorks| Accessibility | © 2021 ThoughtWorks, Inc.