There is one important way of determining whether software you’ve delivered is successful: its acceptance by end users. The best way to ensure this is through rigorous quality processes and practices, something that can only be done by a sharp and attentive QA. But what does it mean to be a good QA? There are many things, but ultimately I believe it comes down to adopting what I call an Agile QA mindset. Below you’ll find some tips and guidelines that will demonstrate what this mindset is and how you can develop it.
1. A QA shouldn’t take pride in detecting a bug: they should be committed to preventing it
Anyone with an Agile QA mindset won’t take pride in detecting a bug during the story testing phase alone. Instead, they should come up with innovative techniques for avoiding a bug in the design or development phase itself. A defect detected at the start of a story, during analysis or design, costs much less compared to the one discovered during the testing phase of the story. It’s for this reason that a QA should be involved from “Sprint 0” when the product is first being structured and framed by research.
2. A QA should never just assume that a feature was implemented correctly
To deliver a quality product, an Agile QA needs to have a mindset in which they are committed to trying to break the system. The QA’s role is not, after all, to simply validate that things are working fine.
QAs need to be both destructive and creative while testing anything. One way of doing this is by creating mind maps for test scenarios; this helps QAs to think beyond the immediate test site and explore the wider implications of a given test and uncover possible branching. An example of a mind map for a login page could be as below.
3. A QA shouldn’t claim a test is complete if they’ve only performed it with a regular use case and test data.
Use cases and test data are the backbone for any sort of testing. Trying with different sets of test data and different combinations in different use cases will allow us to gain greater confidence in the feature being tested. In my experience, I’ve only ever seen issues happen when special types of data inputs — what might be considered “corner cases” — are used.
These include examples like:
A bug occuring only when an unrelated process is running in the background.
- Multiple users trying to upload large files at the same time.
4. A QA should always have a ready reckoner for items they need to refer to regularly
A ready reckoner is a readymade handbook that eases calculations and provides quick references. I always used to have a test ready reckoner with all the following items:
Data samples
Complex algorithms
Often-used SQL queries
Testing tips and tricks
Tricks for setting up (and cooking up) data
Log analysis helpers (like search strings)
Processes for injecting errors
Frequently used Postman collections
Batch job scripts like adding 10 million SQL rows, for example
The ready reckoner is useful because it ensures I don't have to rewrite or recompute something. The important thing to remember here is that these need to be specific to the current project that is under development — they shouldn’t be reused across projects.
5. A QA shouldn’t have to redo things over and over again
Whether you’re executing tests or finding test data, it’s always advisable to have ready-made automated tests, data fetch helpers, or data setup helpers. This will ultimately save you time, money and effort.
Consider a project where there are no automation tests as safety nets. This will cause huge delays in releases as the manual regression cycle will inevitably take longer. This will also lead to less confidence in the code you are pushing to production. For this reason, it’s always advisable to automate all routine tasks.
At the same time, a QA also needs to make sure that nothing is over-engineered. Return on investment is a key consideration when you are planning to automate something.
6. A QA should never waste productive agile team time
Have you ever been in a situation where, while showing a developer something that you had reported as not working, it suddenly starts working? I’ve had this scenario multiple times in my career; it should always remind us as QAs that we need to make sure whatever problem we’ve uncovered hasn’t already been solved by ongoing development work. These fall under the not reproducible in latest build category. To avoid such instances, the test environment should always be on the latest build. CD (Continuous Delivery) should solve this problem.
A bug raised might turn out to be a data issue. To avoid this, we need to have sanitized data in test environments. Similarly, we also need to be aware of infrastructure and environment issues. While they’re a little different — as they could be caused by lower resources in development/test environments — we nevertheless need to be sure that we will not hit this same issue in the Production environment.
These problems mean it’s good sense to double or even triple check issues you've identified to avoid adding unproductive fix time for the team; this will enable faster delivery. At the same time, of course, we still need to be vigilant to not let a real bug escape.
7. A QA should cherish their Cross Functional Requirement (CFR) tests as much as their Functional tests
When trying to test anything, we need to make sure that tests are run on the same platform or testbed as the ones where software used by end-users will run. For a business to customer application (B2C), settings may be multi-dimensional. We might have to test on different mobile software versions, different screen sizes and user profiles. For a business to business application (B2B), this could be as simple as making the test system congruent with the settings dump of the configured system of users.
It’s the QA’s duty to also check the user journey and user experience of the application. Some examples of metrics could be:
How many pages / clicks a user has to go through for placing an order?
Do we have value-adds for enhancing business like “catchy UX to select desserts while ordering dinner?”
Does it meet accessibility standards?
Does it all come together from an aesthetic perspective?
Of the CFRs, user experience, performance, security and compatibility are most important. Depending on the needs of your end users, other CFRs will also add up to this list. It’s also advisable to shift your CFR tests left (ie. upstream) as much as possible.
8. A QA shouldn’t rely on clearing the cache or testing in incognito
There are times when an application will start working again once the cache has been cleared or it has been tried in incognito mode. However, we need to remember that many (probably most!) don’t use the application in incognito mode. That means we need to make sure that the application works well in any mode — whether that’s normal or incognito or cached or uncached.
9. A QA should never take the production environment for granted
If you’ve worked as a QA, you’ve probably heard someone say “This issue occurs only in production. The functionality seems to work fine in lower environments/my machine”. We had exactly this in one particular project when we released an MVP. The cause of the issue was unclear — after some investigation we managed to drill it down to:
The way production was configured, like upstream/downstream systems not being set or added, but added for lower environments. (A quick check of the config constants by QAs can help prevent this.)
Production having a different ecosystem (due to another dependent system’s behavior). This can be mitigated by keeping all upstream/downstream at the same software version as the ones that are going to be released.
We need to make sure that upstream/downstream software systems of lower environments are configured in the same way as the ones that are going to be in production. Getting the infrastructure of lower environments to align with production is going to be very expensive; this means it is worth having at least one environment — say the pre-production environment — almost matching production.
10. A QA must understand that Quality is the entire team’s responsibility
It’s essential to recognise that quality is the responsibility of the whole team — not just the QA.. This mindset is vital as it ensures that everyone has an eye on quality right from the initial design phase until it’s released to the customer. After the release we should also monitor the way the feature or application is being used using instrumentation and user surveys.
Conclusion
For a QA in an agile project, the first thing to do is to build the right mindset. While we’ve looked at 10 key rules or guidelines, it can be summarized as three key elements:
Test early and often
Automate everything that is feasible
Team work
By following those three points and adhering to the 10 tips and tricks above we will be well set to create amazing and high quality applications.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.
