Gradle build optimization: from five minutes to one minute (part two)

In the first part of this article, we explored a solution for optimization by caching the dependencies. We’ll discuss two more solutions from other perspectives in this part. 

2. Eliminating Unnecessary Gradle Tasks

Although we have reduced our build time by caching the dependencies, the nearly empty project still needs more than one minute to build. It only includes three test classes with ten test cases. As the project gets larger, the build time will increase rapidly. To further decrease the build time, we started the analysis on gradle tasks. Gradle provides a profile report feature, which can display the detailed time consumption on each task. We executed ./gradlew clean build --profile locally, and got the result below

In the report we noticed a lot of tasks that we don’t need, such as distTar, bootDistTar, distZip, bootDistZip. After checking each task, what remained was the following: lintKotlin, test, jacocoTestCoverageVerification, jacocoTestReport, bootJar. After eliminating the unnecessary tasks, our build step is eight seconds faster. Moreover, checking the test report generation task is also important because removing the unnecessary report generation will help as well.

3. Improving Test Efficiency

Another optimization point for a general build step is the test execution. Real unit tests are very fast, while those requiring the startups of the whole framework, such as Spring, are slow, because the startup of the framework is slow. We call these tests integration tests, as opposed to unit tests. Actively maintaining the test quantity according to the test pyramid is the first and foremost step of optimizing test efficiency. What we can do more is to reduce the start time of Spring by reusing the test application context.

Are you still satisfied with the sliced startup of Spring at the hands of @WebMvcTest, @DataJpaTest, thinking that these annotations speeded up your tests? Actually, we can still do better. If you take a glance at Spring Boot test context caching, you will realize that Spring tests are fairly clever. They will reuse the ApplicationContext if nothing changed on it during the previous test.

Then how can we maximize the benefits of context caching? The answer is to try to avoid changing the ApplicationContext by annotations such as @MockBean, @SpyBean, @DirtiesContext, @TestPropertySource, @DynamicPropertySource.

For this purpose, when checking our existing tests, we found we had @SpringBootTest, @WebMvcTest, and @DataJpaTest. When we executed the tests the logs showed that Spring framework was started three times, which took a lot of time. In order to reuse ApplicationContext, we changed them to @SpringBootTest (change #1), to avoid restarting Spring during different tests. Then we found we had @MockBean. Our solution is to add the mocked bean to a test configuration file (change #2), so that they are the same for every integration test, and the ApplicationContext was kept the same in different tests. 

Before change:

@WebMvcTest
class ATest(@Autowired val mvc: MockMvc) {
    @MockBean
    lateinit var bService: BService
    @Test
    fun `blahblah`() {
        ...
    }
}

After change:

@SpringBootTest  // change #1
@AutoConfigureMockMvc  // change #1
class ATest(
    @Autowired val mvc: MockMvc,
    @Autowired val bService: BService // change #2
) {
    @Test
    fun `blahblah`() {
        ...
    }
}
@Configuration // change #2
class TestConfiguration {
    @Bean
    fun bService(): BService {
        return Mockito.mock(BService::class.java)
    }
}

After the changes, we executed the tests. The logs showed that Spring was only started once, which was exactly what we were expecting. The profile report also proved our theory since the test time was reduced by three seconds. If you have a lot of integration tests and find it hard to keep them all in a same ApplicationContext, extracting a @IntegrationTest annotation and define all the context-related things in the annotation, and use the annotation in different tests, it will be easier to control.

After all these improvements, we pushed our code to execute it on the CI agent. Our gradle build step consumed only fifty-eight seconds. Adding all the cache pushing and pulling, it took one minute thirteen seconds.

Conclusion

We reduced the CI build time on our gradle build step, from five minutes to one minute, through four methods:

1. Caching the downloaded dependencies by mounting gradle cache.

    

2. Sharing cache files among CI agents to maximize the benefits of caching.

    

3. Eliminating unnecessary gradle tasks to reduce gradle build time.

    

4. Trying to reuse ApplicationContext in integration tests to reduce the startups of Spring framework.

    

    

The official gradle website can provide other tips on performance.