Enable javascript in your browser for better experience. Need to know to enable it? Go here.
Eighteen characteristics of a successful quality strategy

Eighteen characteristics of a successful quality strategy

Quality strategy is integral to product success

 

As a consultancy, most of our engagements involve delivering a bespoke product or transformation of software practices and processes. To consistently exceed our customers’ expectations, we believe quality is an integral part of the changes. And that means, we need a quality strategy that details our test approach for developing and delivering a high quality product to the customers. 

 

And therein lies the challenge: the times we start a project where the complete test approach is crystal clear are rarer than hens’ teeth. The strategy evolves as we get deeper into the context and learn about the business and the product.

 

Testing strategies evolve as projects move from ideation to delivery
Testing strategies evolve as projects move from ideation to delivery

The principles of a robust quality strategy

 

Given our quality strategy cannot be set in stone at the outset — but rather has to evolve with the project — we need some guiding principles that increase our chances of success. We define these as follows:

 

  • The quality strategy artefact should be a reflection of the approach to quality being followed by the team. Hence, it is important everyone buys into these decisions. This helps in avoiding any clashes or a siloed working approach 

  • The strategy should talk about the value of “why we are doing what we are doing” which helps in meaningful conversations driving towards an outcome. Also when someone comes on board to the team, it is vital for them to go through the artefact and understand the value behind our approaches

  • Usually anyone in the team wearing the quality hat is responsible for updating the strategy document and everyone in the team has to contribute to the decisions

  • Just like an Architecture Decision Record, the quality strategy should record decisions whenever they are made

  • Evolution is key here; drafting the strategy gets the engagement off to a good start but over time the document gets stale and people might forget it even exists

  • Visibility of the document is necessary, not just to the team, but to the stakeholders and the business to build confidence in our testing approach   

 

Quality strategy: the essential heuristics

 

Essentially, a quality strategy can cover a wide range of areas. Here, I am classifying them in three broad areas, not necessarily in any order.

 

A quality strategy depicted as three phases: manage, design and deploy
A quality strategy depicted as three phases: manage, design and deploy

Manage

 

  • Vision
    Capture the team’s vision statement for the product. It is important that the business vision and the testing vision are aligned

     

  • Test coverage 
    It is important to set expectations of what’s inscope and out of scope for testing from both a program and a task perspective. Make these visible to various stakeholders. This helps in clearly defining the risks, gaps and responsibilities between different teams and associated systems

     

  • Dependencies
    Through various architectural and business conversations, identify and document dependencies that will directly or indirectly impact your team. These could be technical, system, process or any other dependencies which may impact development and testing

     

  • Number of people required for testing
    In large enterprises where complex systems are being developed, there may be a need for specific people to focus on different testing activities. The strategy may require to understand the number of people required so that there is an  estimate of cost and investment

     

  • Metrics and reporting
    The business vision is vital to identify the metrics we would like to report. Some of the metrics can be empirical (e.g. number of registrations) and some non-empirical (e.g. customer satisfaction). There could also be some internal team metrics such as the four key metrics. These metrics could be used as a base to understand where to focus on testing to improve quality of the application

     

  • Definition of done
    To most organizations, definition of done could be a working feature deployed and delivered to end users. For some others it could also be an exit criteria for moving tasks from one stage to another before production deployment. Document what definition of done means for your team and set those expectations

     

  • Responsibility for writing and maintaining tests
    Though we like to work as a team, once we identify the types of tests to be written, it can help if we make the accountability and responsibility explicit in the document even if we later adapt as required 

     

  • Defect management process and lifecycle 
    This is about the team’s understanding of handling defects including raising, prioritising and fixing them. Also decide on the stages and ceremonies a defect would go through for completion

     

  • Assumptions
    Capture all the assumptions being made alongside the stakeholders on the basis of which, the testing will happen. Validate the assumptions periodically

     

  • Glossary
    Capture all the terminologies from a technical and business perspective for easy reference. This is useful for any new joiners on the team

 

Design

 

  • Types of testing
    Ideally an architecture diagram would help identify the different types of functional tests required such as unit, service / component, contract, integration, system (end to end), exploratory tests
    Have a discussion with representatives from various functions to understand and prioritize cross functional requirements such as security, performance, accessibility, usability and 30+ other ‘ilities’

     

  • Test data
    Document the way test data will be processed and used across various environments. Identify if data protection policies will allow production data versus synthetic data. Data flow between different services may affect upstream and downstream systems. In large programs with external dependencies, it’s also important to understand who will provide what sort of data and timelines

     

  • Tools selection
    Choose your tools for test automation, test management, defect management and document the decisions behind it in the strategy artefact. It is advisable to have the test tools and developer tools use the same language for seamless working and skills transfer

     

Deploy

 

  • Tests to be run on environments
    Sketch out a path to production and map the types of tests you would run in each of these  environments right from your local machines to production

     

  • Infra configurations on different environments
    Most of the time, discrepancy of configurations across environments causes functionality to fail in that environment, such as pointing to the correct database or fetching a particular version of style objects. It is hence essential to document mitigation factors for this risk

     

  • Testing migrations
    If your product development requires data migrations, think of ways to test the migration and its impact on other dependency services. Migration testing may also include testing for backward and forward compatibility

     

  • Testing your deployment process
    Have a backup plan beforehand in case something goes wrong with the deployments to downstream environments. Think about ways you can bring your system in a usable state as quickly as possible after deployments through rollback and roll forwards 

     

  • Monitoring
    Monitoring could come in various forms such as monitoring the hardware and  services. Security and performance of the system can also be monitored. Identifying the right parameters to be monitored is as important as monitoring them

 

This list may not be exhaustive but at least it could be a good starting point for someone to start thinking of the different areas to cover, while writing a quality strategy. Think about which of these would have the biggest impact or make the biggest change and prioritize them accordingly.


Note: Usually these strategy artefacts are pretty long documents either in a Word document or in a slide deck. There’s nothing wrong with those formats as long as the team is happy. But in case you fancy creating a one page holistic quality strategy, here is another way of looking at it.

Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.

Explore the latest Technology Radar