ThoughtWorks
  • Contact
  • Español
  • Português
  • Deutsch
  • 中文
Go to overview
  • Engineering Culture, Delivery Mindset

    Embrace a modern approach to software development and deliver value faster

    Intelligence-Driven Decision Making

    Leverage your data assets to unlock new sources of value

  • Frictionless Operating Model

    Improve your organization's ability to respond to change

    Platform Strategy

    Create adaptable technology platforms that move with your business strategy

  • Experience Design and Product Capability

    Rapidly design, deliver and evolve exceptional products and experiences

    Partnerships

    Leveraging our network of trusted partners to amplify the outcomes we deliver for our clients

Go to overview
  • Automotive
  • Cleantech, Energy and Utilities
  • Financial Services and Insurance
  • Healthcare
  • Media and Publishing
  • Not-for-profit
  • Public Sector
  • Retail and E-commerce
  • Travel and Transport
Go to overview

Featured

  • Technology

    An in-depth exploration of enterprise technology and engineering excellence

  • Business

    Keep up to date with the latest business and industry insights for digital leaders

  • Culture

    The place for career-building content and tips, and our view on social justice and inclusivity

Digital Publications and Tools

  • Technology Radar

    An opinionated guide to technology frontiers

  • Perspectives

    A publication for digital leaders

  • Digital Fluency Model

    A model for prioritizing the digital capabilities needed to navigate uncertainty

  • Decoder

    The business execs' A-Z guide to technology

All Insights

  • Articles

    Expert insights to help your business grow

  • Blogs

    Personal perspectives from ThoughtWorkers around the globe

  • Books

    Explore our extensive library

  • Podcasts

    Captivating conversations on the latest in business and tech

Go to overview
  • Application process

    What to expect as you interview with us

  • Grads and career changers

    Start your tech career on the right foot

  • Search jobs

    Find open positions in your region

  • Stay connected

    Sign up for our monthly newsletter

Go to overview
  • Conferences and Events
  • Diversity and Inclusion
  • News
  • Open Source
  • Our Leaders
  • Social Change
  • Español
  • Português
  • Deutsch
  • 中文
ThoughtWorksMenu
  • Close   ✕
  • What we do
  • Who we work with
  • Insights
  • Careers
  • About
  • Contact
  • Back
  • Close   ✕
  • Go to overview
  • Engineering Culture, Delivery Mindset

    Embrace a modern approach to software development and deliver value faster

  • Experience Design and Product Capability

    Rapidly design, deliver and evolve exceptional products and experiences

  • Frictionless Operating Model

    Improve your organization's ability to respond to change

  • Intelligence-Driven Decision Making

    Leverage your data assets to unlock new sources of value

  • Partnerships

    Leveraging our network of trusted partners to amplify the outcomes we deliver for our clients

  • Platform Strategy

    Create adaptable technology platforms that move with your business strategy

  • Back
  • Close   ✕
  • Go to overview
  • Automotive
  • Cleantech, Energy and Utilities
  • Financial Services and Insurance
  • Healthcare
  • Media and Publishing
  • Not-for-profit
  • Public Sector
  • Retail and E-commerce
  • Travel and Transport
  • Back
  • Close   ✕
  • Go to overview
  • Featured

  • Technology

    An in-depth exploration of enterprise technology and engineering excellence

  • Business

    Keep up to date with the latest business and industry insights for digital leaders

  • Culture

    The place for career-building content and tips, and our view on social justice and inclusivity

  • Digital Publications and Tools

  • Technology Radar

    An opinionated guide to technology frontiers

  • Perspectives

    A publication for digital leaders

  • Digital Fluency Model

    A model for prioritizing the digital capabilities needed to navigate uncertainty

  • Decoder

    The business execs' A-Z guide to technology

  • All Insights

  • Articles

    Expert insights to help your business grow

  • Blogs

    Personal perspectives from ThoughtWorkers around the globe

  • Books

    Explore our extensive library

  • Podcasts

    Captivating conversations on the latest in business and tech

  • Back
  • Close   ✕
  • Go to overview
  • Application process

    What to expect as you interview with us

  • Grads and career changers

    Start your tech career on the right foot

  • Search jobs

    Find open positions in your region

  • Stay connected

    Sign up for our monthly newsletter

  • Back
  • Close   ✕
  • Go to overview
  • Conferences and Events
  • Diversity and Inclusion
  • News
  • Open Source
  • Our Leaders
  • Social Change
Blogs
Select a topic
View all topicsClose
Technology 
Agile Project Management Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionary Architecture Experience Design IoT Languages, Tools & Frameworks Legacy Modernization Machine Learning & Artificial Intelligence Microservices Platforms Security Software Testing Technology Strategy 
Business 
Financial Services Global Health Innovation Retail  Transformation 
Careers 
Career Hacks Diversity & Inclusion Social Change 
Blogs

Topics

Choose a topic
  • Technology
    Technology
  • Technology Overview
  • Agile Project Management
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionary Architecture
  • Experience Design
  • IoT
  • Languages, Tools & Frameworks
  • Legacy Modernization
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Platforms
  • Security
  • Software Testing
  • Technology Strategy
  • Business
    Business
  • Business Overview
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Careers
    Careers
  • Careers Overview
  • Career Hacks
  • Diversity & Inclusion
  • Social Change
Software TestingTechnology

Changing the Metrics Conversation

Bruce McLeod Bruce McLeod

Published: Jan 14, 2016

In the traditional world of enterprise software delivery, there can be a strong focus on software delivery metrics - What’s the test coverage? What is the code coverage? What’s the bug count? Why is the bug count so high? How many requirements have we delivered in this release?

Workshop

Without the context that surrounds them, these types of measures are completely nonsensical. I could answer all the questions above, yet still not provide any meaningful information. For example, the answers could be just 81%, 54%, 134, schedule pressure, 34.

Whilst this may answer all the questions, it still fails to provide any value to the conversation on software delivery.

Changing the Conversation

When faced with questions like the above, I would approach them from a different angle. I would seek to understand why these questions are being asked, and if they are indeed the right questions. Are they trying to track test execution progress? Are they trying to track ‘quality’? Are they trying to get a degree of comfort that the questioner will be able to answer any question that is asked of them?

The Importance of Understanding

Approaching this type of discussion with empathy is important. It is very easy to come across in this situation in a negative way - “That’s a stupid metric”, “Code coverage means nothing”, “Why are you asking for something that adds no value and making us do more work to get it?”. Instead, understand the reasoning behind the questions - the real answer may be that stakeholders may not have enough visibility into your work, or have the confidence that your current process is going to produce a quality result.

Testing is Never Complete

Personally, I find that most non-testers assume, subconsciously and incorrectly, that testing is complete. To counter that, I would like to give an example of a six character user name into a textbox limited to ASCII input. This simple example still requires 127 x 127 x 127 x 127 x 127 x 127 = 4,195,872,914,689 test cases. Even if each test case takes us only 1 second to execute, it will still take 7.9 million years to finish testing the user name field, before we move onto the password field.

Something else I would like to discuss is Cyclomatic Complexity as an indicator of how to achieve a representative level of statement coverage. Once I have laid the foundation with those two points, I can then evolve the conversation to  directly address their initial concerns. For example:

  • Arriving at a count of test cases needed to achieve test coverage goals, for the sake of chasing a number, which is the incorrect thing to do
  • Most people create flaws in test coverage numbers
  • Any claim of a high level of test coverage, without a proportionately high number of unit tests is probably incorrect
  • Counting test cases, bugs and requirements is a lot like counting fruit. You may have a grape and a watermelon, but they are still treated as being the same size in a metrics context.

With improved understanding and hopefully trust, the conversation should be a more productive one.

Proactive Radiating of Information

The other key thing I’d like to recommend is to proactively radiate information, as opposed to reactively defend its absence. This allows me to focus my attention and energy on shipping great software.

Hopefully, your conversation has moved back to what actually is to be measured, what your stakeholders’ concerns are, who the target audience is for the “numbers” and how you can help them tell their story better.

  • What we do
  • Who we work with
  • Insights
  • Careers
  • About
  • Contact

WeChat

×
QR code to ThoughtWorks China WeChat subscription account

Media and analyst relations | Privacy policy | Modern Slavery statement ThoughtWorks| Accessibility | © 2021 ThoughtWorks, Inc.