This blip is not on the current edition of the Radar. If it was on one of the last few editions it is likely that it is still relevant. If the blip is older it might no longer be relevant and our assessment might be different today. Unfortunately, we simply don't have the bandwidth to continuously review blips from previous editions of the RadarUnderstand more
Published: Apr 05, 2016
Last Updated: Oct 28, 2020
Oct 2020

Traditional testing approaches focus on evaluating if our production code is doing what it's supposed to do. However, we could make mistakes in the testing code introducing incomplete or useless assertions that create a false sense of confidence. This is where mutation testing comes in; it assesses the quality of the tests themselves, finding corner cases that are hard to realize. Our teams have used Pitest for a while now, and we recommend its use in Java projects to measure the health of the test suite. In short, mutation testing introduces changes in the production code and executes the same tests a second time; if the tests are still green it means that the tests are not good and need to improve. When you’re using programming languages other than Java Stryker is a good choice in this space.

Nov 2016

Pitest is a test coverage analysis tool for Java that uses a mutation-testing technique. Traditional test coverage analysis tends to measure the number of lines that are executed by your tests. It is therefore only able to identify code that is definitely not tested. Mutation testing, on the other hand, tries to test the quality of those lines that are executed by your test code and yet might contain general errors. Several problems can be spotted this way, helping the team to measure and grow a healthy test suite. Most of such tools tend to be slow and difficult to use, but Pitest has proven to have better performance, is easy to set up, and is actively supported.

Apr 2016