Master
ThoughtWorks
Menu
Close
  • What we do
    • Go to overview
    • Customer Experience, Product and Design
    • Data Strategy, Engineering and Analytics
    • Digital Transformation and Operations
    • Enterprise Modernization, Platforms and Cloud
  • Who we work with
    • Go to overview
    • Automotive
    • Healthcare
    • Public Sector
    • Cleantech, Energy and Utilities
    • Media and Publishing
    • Retail and E-commerce
    • Financial Services and Insurance
    • Not-for-profit
    • Travel and Transport
  • Insights
    • Go to overview
    • Featured

      • Technology

        An in-depth exploration of enterprise technology and engineering excellence

      • Business

        Keep up to date with the latest business and industry insights for digital leaders

      • Culture

        The place for career-building content and tips, and our view on social justice and inclusivity

    • Digital Publications and Tools

      • Technology Radar

        An opinionated guide to technology frontiers

      • Perspectives

        A publication for digital leaders

      • Digital Fluency Model

        A model for prioritizing the digital capabilities needed to navigate uncertainty

      • Decoder

        The business execs' A-Z guide to technology

    • All Insights

      • Articles

        Expert insights to help your business grow

      • Blogs

        Personal perspectives from ThoughtWorkers around the globe

      • Books

        Explore our extensive library

      • Podcasts

        Captivating conversations on the latest in business and tech

  • Careers
    • Go to overview
    • Application process

      What to expect as you interview with us

    • Grads and career changers

      Start your tech career on the right foot

    • Search jobs

      Find open positions in your region

    • Stay connected

      Sign up for our monthly newsletter

  • About
    • Go to overview
    • Our Purpose
    • Awards and Recognition
    • Diversity and Inclusion
    • Our Leaders
    • Partnerships
    • News
    • Conferences and Events
  • Contact
Global | English
  • United States United States
    English
  • China China
    中文 | English
  • India India
    English
  • Canada Canada
    English
  • Singapore Singapore
    English
  • United Kingdom United Kingdom
    English
  • Australia Australia
    English
  • Germany Germany
    English | Deutsch
  • Brazil Brazil
    English | Português
  • Spain Spain
    English | Español
  • Global Global
    English
Blogs
Select a topic
View all topicsClose
Technology 
Agile Project Management Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionary Architecture Experience Design IoT Languages, Tools & Frameworks Legacy Modernization Machine Learning & Artificial Intelligence Microservices Platforms Security Software Testing Technology Strategy 
Business 
Financial Services Global Health Innovation Retail  Transformation 
Careers 
Career Hacks Diversity & Inclusion Social Change 
Blogs

Topics

Choose a topic
  • Technology
    Technology
  • Technology Overview
  • Agile Project Management
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionary Architecture
  • Experience Design
  • IoT
  • Languages, Tools & Frameworks
  • Legacy Modernization
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Platforms
  • Security
  • Software Testing
  • Technology Strategy
  • Business
    Business
  • Business Overview
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Careers
    Careers
  • Careers Overview
  • Career Hacks
  • Diversity & Inclusion
  • Social Change
Software TestingBangaloreTechnology

6 Ways to Speed Up Your Tests

Devi Sridharan Devi Sridharan

Published: Aug 5, 2014

When you’re driven by the enthusiasm of having a good test coverage for your application, you automate the entire workflow**. As a result of this, the test-suite becomes huge and the safety net of these tests become a bottleneck. What starts off as a method to give faster feedback, now becomes a cause for delaying code commits.

This is a common challenge faced by most teams. We faced a similar situation, but we overcame this challenge by taking advantage of the automated test suite. Here’s how we did it:

#1 - Identify flaky tests

What are flaky tests?  

Let’s take a typical scenario from our projects…it’s been a couple of hours and you’re still waiting for a successful build to commit your changes to. Not wanting to delay your commit further, you decide to fix these tests yourself. To your amazement, you discover that all the tests are passing in re-run. Tests that fail randomly and pass on re-run without code changes are known as ‘Flaky Tests’.

As Martin says in his blog, these flaky tests can take away the benefits of your bug detector(automated test suite). Flakiness in a test-suite can completely break confidence in the test-suite. Benefits of the automated test-suite can be enjoyed only when it’s reliable and gives timely alerts.

Therefore, our first task at hand is to get a green build and the confidence back. As Martin suggests, follow this workflow:

Work flow

By now, you’ll probably have a green build and you can commit your changes. Any test failing hereafter will be for genuine reasons.  

#2 - Break the tests

Another way to quickly get feedback from tests is to run them in multiple categories. Break the tests into categories according to the functionality or by nature of the  tests -

  • Smoke test - the tests needed to ensure that critical functionalities(i.e business & technical behaviors) are intact.

  • Regression tests - the tests which cover all other workflows.

A smoke test could be executed as part of every commit, and regression tests could be executed as multiple commits or on a timely basis. In this manner, we could run only the needed subsets on every commit. This will make sure that the commits aren’t delayed and there is no compromise on tests.

If categorising the tests is a lot to do, there is always an alternative solution - parallelisation.

Parallelisation: There are lot of parallelisation tools available in the market. If you have the luxury of multiple VMs and test parallelisation tools like Selenium grid and Test Load Balancer(TLB), the execution time can be brought down drastically.

Once the delayed commits are taken care of, we can actually start looking at ways to improve the tests. The methods suggested above can bring in some immediate relief, but it isn’t permanent.

# 3 - Create test data smartly

A good automation test is the one which creates and cleans up the needed data after use. 

By following this rule of thumb for good automation, we would have ended up with tests which spend more time creating the test data rather than testing it! We can't compromise on good practices in order to have faster builds, can we?

Most of the automated tests create test data through a browser. It would be very effective if we create test data directly in the database. DB calls are cost and time efficient. By reducing the UI interaction, the test becomes fast and more reliable. However, if projects have complex DB interactions then this approach may not be handy.

#4 - Think before you try xpaths

I used to be a fan of xpaths. Most of us fail to realise that locator selectors contribute  to the execution speed of tests in a big way. When xpaths are used for object identification, a lot of the background processing/filtering happens. Refer this to understand more about how xpaths are processed.

Xpath identification is generally slower than other forms of object identifications. Therefore, it always helps to identify objects with direct attributes like ID, Name, etc. You can use CSS selectors as an alternative. But try and keep xpaths as your last saviour. The result of these locator strategies can be seen gradually over the period of time.

#5 - Add more caffeine to tests

Wait statements are added in tests when we navigate between pages, or when we wait for the asynchronous call to finish.These wait statements are deadly as they make tests slow, are less reliable and at times flaky/brittle.

Try to consciously eliminate wait statements in your tests. Replace all generic wait statements with conditional waits (such as the wait for a page load to finish or an element to appear etc). Conditional waits will then wait only for a specific action/event to occur and proceed with execution as soon as it's complete. 

#6 - Leverage the efficiency of Unit-Tests

The test pyramid tells us to add more tests to the lower layer of the pyramid and less at the end-to-end layer. Tests can be identified at unit, integration and UI levels.

Tests at the lower level of the pyramid are faster than UI tests. Identify a test which doesn't necessarily need a browser. Actions like verifying the contents of a downloaded file or email can be tricky and time consuming when tested in a browser. Unit tests can handle these functions efficiently with the help of mocks and stubs.

Testing triangle

Identifying and pushing tests to a lower level makes tests faster and reliable. Join hands with developers, review the unit-tests and add more to the unit-levels. By reviews, you can avoid test duplication in multiple layers. Add integration test, if applicable. Ensure only high-level workflows are covered in end-to-end/UI tests.

These nifty ways helped my team overcome the side-effects of a large test-suite and reap all benefits of an automated test-suite. I'm sure there are more, so share your ideas with us! 

workflow** - From a more abstract or higher-level perspective, workflow may be considered a view or representation of real works.In terms of testing,scripted end-to-end testing which are expected to be utilized by end-user.

 

 

Master
Privacy policy | Modern Slavery statement | Accessibility
Connect with us
×

WeChat

QR code to ThoughtWorks China WeChat subscription account
© 2021 ThoughtWorks, Inc.