Testing
Asking questions to improve quality
As more companies use KPIs (Key Success Indicators) to measure their software quality rate, new opportunities emerge for quality analysts. There are many possibilities for a QA, like when we are placed in teams or organizations with poorly defined practices or where the role of quality analysts is not understood yet.
We know that the activities developed by quality analysts depend on context, but there is a method that we can use to easily visualize points of improvement, which will quickly add value to the company and the team. So, when we accept the opportunity to work in a team that does not have well-defined quality processes, we need to be ready to answer the following two questions: "How are you going to add value to the team?" and "What activities are you going to perform?"
The answer will usually be the same: understand the context and identify points of improvement. In this way, you can work to be strategic in your actions. Because any statement that is not based on facts and data, in addition to showing that it was not a team decision, has high chances of not being achieved.
Besides, pairing creates virtuous exchanges. This practice will accelerate the understanding of the technologies used in the project and in the company.
This step creates inputs to identify the points that we can add value to with less effort.
The next step will be to add value by solving the most evident everyday problems that require frequent rework. The strengths to be improved will always be those that require a greater investment of effort and time, but they can harm the product or business if they are not accomplished.
You can guide your actions based on the following points:
With these practices, you will have more space to propose changes and new solutions, making a significant contribution to delivery quality.
"What is not measured cannot be improved." So, when we think about software quality, measuring impact is important to know if we are on the right track. Otherwise, we can change the strategy to align with the business objectives.
The metrics to be mapped will depend on which goals were listed to be achieved, such as code coverage rate, defects found in production, and cyclomatic complexity.
Through metrics, we can demonstrate the reliability and consistency of the work developed, using this as an input to make strategic decisions.
This is a crucial moment in our cycle. Through presentation, we demonstrate our success rate using the indicators listed at the beginning of the planning. At this point, we must present the goals carried out within the time frame, showing the result of the impacts caused by the team's action plan.
In addition to the points worked on, we must keep in mind the things we learned from the results and the next steps to be developed in the next cycle.
Between interactions, she receives the information that her team has been tasked with carrying out an increase in the project’s quality rate, and the first objective is to have a 30% code coverage. When talking to the team, Isabelle realizes that, currently, tests are not carried out on any application layer. She perceives a misalignment between the KPI established by the team's management and their attitude towards it.
Isabelle discovers that, at the beginning of the project, a developer named Fernanda started to create unit tests and did not continue because of the delivery demand and lack of engagement from the rest of the team.
Thus, the team agrees to start developing unit tests for the module responsible for clearing payments. So, Isabelle and Fernanda are responsible for starting the activity in the next development cycle.
As soon as Isabelle and Fernanda start creating the first tests, they realize that, when testing in the periods of the year that were previously attributed to the beginning and end of daylight saving time, the system behaves in an unexpected way. They end up finding out that there was an inconsistency in the system, which was still considering daylight saving time in its logic.
With the presentation, more team members became aware of the importance of testing in the project, so they estimated development time, including tests.
We can conclude that, in order to align the result of the software quality process with the proposed KPIs and stakeholders' expectations, we need to expose the process that will be used. In this way, we are able to receive feedback to improve actions with each new cycle we run. When we know where we want to go, reaching the goal is just a matter of time.
We know that the activities developed by quality analysts depend on context, but there is a method that we can use to easily visualize points of improvement, which will quickly add value to the company and the team. So, when we accept the opportunity to work in a team that does not have well-defined quality processes, we need to be ready to answer the following two questions: "How are you going to add value to the team?" and "What activities are you going to perform?"
The answer will usually be the same: understand the context and identify points of improvement. In this way, you can work to be strategic in your actions. Because any statement that is not based on facts and data, in addition to showing that it was not a team decision, has high chances of not being achieved.
To support this improvement process, I will introduce you to the AIMA (Analysis, Impact, Metrification, and Presentation) method:
Analysis
In this first step, collect as much information as possible and take notes. By intentionally observing, we can identify improvement points, which can be found through any interaction with the team (e.g. meetings or ceremonies).Besides, pairing creates virtuous exchanges. This practice will accelerate the understanding of the technologies used in the project and in the company.
This step creates inputs to identify the points that we can add value to with less effort.
Impact
The next step will be to add value by solving the most evident everyday problems that require frequent rework. The strengths to be improved will always be those that require a greater investment of effort and time, but they can harm the product or business if they are not accomplished.You can guide your actions based on the following points:
- Create trusting relationships with your team;
- Put the experience of the end user at the center of decision-making;
- Discuss the chosen KPIs;
- Align expectations with the project stakeholders whenever necessary.
With these practices, you will have more space to propose changes and new solutions, making a significant contribution to delivery quality.
Metrification
"What is not measured cannot be improved." So, when we think about software quality, measuring impact is important to know if we are on the right track. Otherwise, we can change the strategy to align with the business objectives.The metrics to be mapped will depend on which goals were listed to be achieved, such as code coverage rate, defects found in production, and cyclomatic complexity.
Through metrics, we can demonstrate the reliability and consistency of the work developed, using this as an input to make strategic decisions.
Presentation
This is a crucial moment in our cycle. Through presentation, we demonstrate our success rate using the indicators listed at the beginning of the planning. At this point, we must present the goals carried out within the time frame, showing the result of the impacts caused by the team's action plan.In addition to the points worked on, we must keep in mind the things we learned from the results and the next steps to be developed in the next cycle.
The AIMA method in practice
Context:
Isabelle was hired by a company in the financial sector and received the information that she would be joining a newly formed team that is responsible for developing new functionality for its main system. The functionality should clear the payment of Boleto bank slips within an hour.Step 1: analysis
After joining the team, Isabelle starts her observations, taking note of everything that may be relevant. When interacting with the team, she asks questions about the new functionality, how the current system works, what are the reasons for the change, what is the deadline for implementation and how the project architecture works.Between interactions, she receives the information that her team has been tasked with carrying out an increase in the project’s quality rate, and the first objective is to have a 30% code coverage. When talking to the team, Isabelle realizes that, currently, tests are not carried out on any application layer. She perceives a misalignment between the KPI established by the team's management and their attitude towards it.
Isabelle discovers that, at the beginning of the project, a developer named Fernanda started to create unit tests and did not continue because of the delivery demand and lack of engagement from the rest of the team.
Step 2: impact
Isabelle talks to Fernanda and presents a testing strategy based on the proposed KPI, starting with a point of great risk that is the module responsible for clearing the Boleto payment within an hour, with the previous term being up to 3 days. With Fernanda's support, Isabelle discusses with the team the importance of carrying out the tests and what its impact would be if there was a failure to clear payments.Thus, the team agrees to start developing unit tests for the module responsible for clearing payments. So, Isabelle and Fernanda are responsible for starting the activity in the next development cycle.
As soon as Isabelle and Fernanda start creating the first tests, they realize that, when testing in the periods of the year that were previously attributed to the beginning and end of daylight saving time, the system behaves in an unexpected way. They end up finding out that there was an inconsistency in the system, which was still considering daylight saving time in its logic.
Step 3: metrification
To measure the level of code coverage achieved, Isabelle and Fernanda use a code coverage tool to generate a report of the percentage of lines covered per test. This practice, in addition to showing what is not covered by tests, allows the team to make strategic decisions by identifying which points are less covered by tests and, with a risk analysis strategy, detecting which areas of the system should be prioritized when creating tests.Step 4: presentation
After the end of the development cycle, Isabelle and Fernanda use the information collected to make a presentation to the stakeholders, demonstrating the evolution concerning the established goal. They demonstrate that they obtained an increase in the coverage rate by 8% and that the next step will be to implement a test step in the routine of the project pipeline.With the presentation, more team members became aware of the importance of testing in the project, so they estimated development time, including tests.
We can conclude that, in order to align the result of the software quality process with the proposed KPIs and stakeholders' expectations, we need to expose the process that will be used. In this way, we are able to receive feedback to improve actions with each new cycle we run. When we know where we want to go, reaching the goal is just a matter of time.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.
