DevOps, a movement of people who care about developing and operating reliable, secure, high performance systems at scale, has always — intentionally — lacked a definition or manifesto. However (and this is fascinating in its own right) that doesn’t mean that we can’t measure the impact of DevOps, or how good people are at doing it. The proof of this, and also of the startling impact of the DevOps movement, is now available in the form of the 2014 State of DevOps report (which you can download for free).
The report, a collaboration between Nicole Forsgren Velasquez, Gene Kim, Puppet Labs, and yours truly, surveyed over 9,200 people worldwide, covering a wide range of industries and types of organization. Our goal for the report was ambitious. We set out to measure IT performance, business performance, the impact of particular practices (such as continuous integration, test automation, and version control), and also culture, and then to discover to what extent they influenced each other. How, you might ask, do you measure these things like culture and organizational performance? Following Douglas Hubbard’s definition of measurement as “A quantitatively expressed reduction of uncertainty based on one or more observations,” it turns out that you can measure anything if you put your mind to it. The report describes both our methodology and the way we measured these apparent intangibles.
Indeed we not only measured these things: we have sound, statistically significant data that shows that culture and DevOps practices impact both IT performance and organizational performance. In direct contradiction to a popular narrative of the last ten years, IT matters — indeed, the results show it is acompetitive advantage — and DevOps culture and practices are instrumental in achieving both high IT performance and organizational performance. Readers of this blog will be especially interested to learn that:
- Trunk-based development, continuous integration, and automated testing measurably improve both IT performance and organizational performance.
- Having a high-trust culture has a strong impact on both IT performance and organizational performance.
- Using external change approval processes such as a change advisory board, as opposed to peer-based code review techniques, significantly impacts throughput while doing almost nothing to improve stability.
- Job satisfaction is the biggest predictor of organizational performance, and using DevOps practices are good predictors of job satisfaction.
I’m very excited by the report. We improved on last year’s method for measuring IT performance. We showed how you can measure culture and organizational performance. Most important, the analysis of our enormous data set demonstrates definitively that the strategies championed by the DevOps movement work, and that they provide a competitive advantage to your business.
Many thanks to my collaborators, the fabulous team at PuppetLabs, and to all of you who took the survey.