AI-assisted software development in 2025

Ken Mugrage: Hello, everybody, and welcome to another edition of the Thoughtworks Technology Podcast. My name is Ken Mugrage. I'm one of your regular hosts. I have with me today, Chris Westerhold. Chris, do you want to give people a quick intro?

Chris Westerhold: Great to get on the podcast for the first time, and it's great to get everybody else. Chris Westerhold. I am a Global Practice Director for our Engineering Excellence Offering and really focus a lot on engineering metrics, engineering insights, platform engineering, driving AI into software delivery lifecycle. It's really great to get on here and chat with you.

Ken: Cool. It's good to have you. Our topic today, we're going to talk about the latest DORA report. I have to admit, for me personally, it's very hard not to call it the state of DevOps report, because as many people probably know, that's really where it came from. Thoughtworks actually was one of the very early supporters and contributors to it, gosh, 10, 15 years ago, whenever it was.

Then it became more of a peer-reviewed scientific thing. DORA was acquired by Google. DORA changed it this year, and it's the State of AI-assisted Software Development report. We're still happy to be actually the platinum sponsor of it. We very firmly believe in solid research-driven metrics. As always, that won't affect our opinion much.

[Laughter]

Chris: That's right.

Ken: I guess, Chris, it's been the gold standard, as I mentioned, for DevOps reports for metrics and things for quite a while. There's things that we use in our everyday terminology that came out of it, but now it's squarely on AI. What's your takeaway? What's your headline, if you had one?

Chris: When I think of what's the key headline from the DORA report is really AI is pretty ubiquitous now. It's not if you're using it, it's really how well are you using it? How do you start to drive that impact into your organization, and how do you measure it? If I get asked one question, it's almost, "How do I measure AI impact? How do I measure AI impact?" I think the DORA metrics do a great job of that, but you need to supplement that with some other things that we'll talk about. It's this holistic kind of framework. I think DORA does a great job of bringing that research together to really give you a roadmap forward.

Ken: One of the things that the DevOps reports did is they compared, they call them the four metrics, but two of them were mean time to repair and mean time to failure. They don't go in the same direction, generally speaking. I felt like they said the same thing about AI, like we can increase throughput, but is that giving benefit? What's your take on that?

Chris: You can absolutely increase throughput, but you can destroy your product at the same time. For an example, I can have my AI assistant go and write 1,000 tests. It's going to extend out my build times, it's going to make it harder to deploy, harder to maintain, and it may not even improve my release confidence. It may make everything a bit more fragile.

Just the act of doing something, that has changed significantly, where before it was, "Oh, we need better testing. We need better testing. We need better testing." Now you can get better testing, but is it the right testing? If you're focused on the wrong metrics, like, "Oh, we need 80% code coverage," yes, on the surface, that sounds okay, but you can get to 80% code coverage with really bad tests that, again, don't help you with release confidence.

Ken: Which drives the question, is it better testing you can get, or is it just more testing?

Chris: That's right.

Ken: If the goal is better testing, which I assume it would be, are you aware of any practices that people can do? We talk about our sensible defaults and so forth, a bunch. Podcast listeners can go look up those episodes. Are there practices that help ensure that it's better, not just more?

Chris: This is where test-driven development can come in. A lot of these practices that we've had before still apply. Leveraging AI takes away the pain of having to type, but you still need to be the one driving. You still need the one thinking through the problem. It depends on what models you're using now, but some of them have a lot of tendencies, for example, to put in fallbacks that basically negate the problem. "Oh, well, this error happened." You say, "Hey, go help me fix this error."

What did it do? It just wrote a fallback to fall back to basically nothing. It made your test pass, and it made you think the problem went away until you actually went into the application and like, "Why doesn't this work? I wrote it. I have tests. I've all the rest of it." Because it actually did something you didn't expect. You need to be continuing the really good processes that strong engineers bring to the table. It just looks a little bit different now than it used to.

Ken: I think it was one of your papers I was reading that said something about, you can code faster, but can you review faster? It's kind of the same thing. How do people protect themselves from it going and changing things like that and them not even knowing about it? Is there something teams can do to watch for it?

Chris: Yes, you can. There's even ways to leverage AI for this. I know we're in a world where agents are everything and they're everything and nothing all at the same time. How do you start to build your own personal workflow of working through something and I've found that I can work on a problem leveraging my favorite AI tool or whatever, finish something. Then you would go back and do that architectural review. You would go back and truly understand what you're trying to accomplish.

You can also leverage some agents to be able to help with that. By coming up with a set of prompts to say, "Hey, how would I want to go through this workflow by myself?" You can now then leverage that workflow with an AI assistant and say, "Hey, I want you to go back and check these things." They'd give me proof that all of this stuff is happening.

Again, you're not the one having to manually go through it. If you've been around long enough and you're paying attention to the patterns and the architecture and what you know about the application, you're going to start to see things pop out really quickly of like, "Oh, wait a minute, what was that?" Now you can start to fix them. Don't fall into the pattern of just believing whatever a model says and whatever a model says you should do because there's a good chance it won't work.

Ken: One of the things that people are pushing towards is a phrase has been around the industry for quite a while now that we've rallied against, but that still goes around the whole 10X devs. What's your take on that? Is this actually making that happen or what are we talking about here?

Chris: I still personally believe that 10X engineers are almost a myth, but a 10X organization, that is a thing. The companies that allow their engineers to be able to easily work and get their job done. That's where you can really start to see the benefits of we have a really clear and easy path to production. We have a clear testing process.

How do we want to go about doing those things? We understand how we want to operate through the software delivery life cycle that unlocks everyone. You have the checks and balances that go into play and be able to allow an engineer to be able to work a lot faster. I know I started that with 10X engineers are a myth. They're not really a myth. They are a thing.

There are some people that are just really good at it and there's people that are doing it because it's a thing that pays the bills and allows them to be able to move forward. Both of those are totally valid, but the goal for an organization is to 10X all of your developers and allow everyone to be able to go. The rising tide lifts all boats mentality and by being able to have a well-structured SDLC, allows for all of that to happen.

Ken: Does that change the structure of the team? The personas they play. Because you hear now that, "Oh, I can do all my business analysis with AI." What's this team structure look like?

Chris: It's still a little bit up in the air, don't get me wrong. I will fall back to something that was still true even before the advent of AI, that for anybody that's ever done a group project, you can do two, you can do three but the moment you go more than that, you now start to have some serious problems where you can have somebody not really doing anything. You need a lot more management. You need a lot more coordination.

All of those things don't don't go away with the bringing AI and those kinds of things into the software delivery life cycle. You just need to now understand how do you want to operate. I will always gravitate towards smaller teams because they need less management. You can give them an outcome, let them drive to it and then get out of the way. AI is now allowing people to own a broader scope than they ever used to.

There might be a relatively smaller team that might be able to manage an application that is in more of a maintenance mode. It's not in hypergrowth or those kinds of things, more of a maintenance type application. It could be a much smaller team because those folks can handle a broader set of things to keep that thing up and running. Where before it would take a much larger team.

Ken: Another concept they talk about in the DORA report, and it reminds me a lot, I'll be honest, again, back to its DevOps roots when we're talking about, if you have junk, then it just gives you faster junk. They introduced a concept called the AI mirror. What's your take on that?

Chris: This is 100% true. AI will give you whatever you are trying to do much faster. If you have bad practices, if you have bad software, just the way you go about writing software, it will just multiply that. If you are creating bad code, if you have bad practices, all the rest of it, it's just going to make it worse. For example, just about everyone has poor documentation. Every developer experience survey I've ever come across, people have poor documentation.

It's always number one, number two, number three. What can you do with AI? You could have it just crank out documents. If you had 10,000 pages in Confluence or SharePoint or whatever your knowledge management tool is, if you created another 100, 300, 500, would it make your documentation problem better or worse? I would say that it's actually going to make your problem worse. What you need to be thinking about is how do I make those questions go away?

You can't be thinking about as a documentation problem, you have to be thinking about it as a question problem, a complexity problem, a coding problem. Then how do you bring in AI tools to be able to say, "Well, how do I start to analyze my current code bases?" Think about building things like abstract syntax trees, knowledge graphs, understanding the dependencies across code bases, and being able to now leverage and build a set of knowledge that you can then interrogate in a better way than trying to read through a whole bunch of confluence documents.

That's the shift that really needs to happen. When I think about that mirror, if I'm just trying to do the things that I did in the past just faster, you're actually not going to get much better and you are really going to start building up massive amounts of tech debt, but if you're doing that, you probably already have a lot anyway.

Ken: You wrote an article recently. For our audience, I'm not sure if it's published yet or not, but you wrote one. It talks about the AI engineering waste. You insert concept in there that's interesting, that the tools that are meant to make us faster actually cause friction. We have to think about that too. Can you talk a little bit about some examples of that?

Chris: Context becomes such a huge thing now. You're only as good as your slowest part, and you have to be able to start to pull together sets of context to allow you to be able to move faster. Just like what I was talking about with the rethinking documentation and building that kind of knowledge around an individual code base, that now becomes context for you to be able to use in your future software development.

You can say, "Oh, I don't need to spend an hour or more digging through an application that I don't know. I can now interrogate it to find exactly what I need, how that all interplays with the other parts of the application, and I now have that context to be confident in what I want to do to be able to move forward."

That is an incredibly powerful set of stuff. Then you now need to take that context and start to build it into that tool chain that I talked about a little bit ago. How do you want to start to go about doing the work? I now understand the complexities of a software application. I now can compare what I want to do versus where things are now. I then can have this AI-assisted ability to write that new code.

Now I can have a number of agents that can go and interrogate and tell me all the things that I did wrong, and be able to now have this really well thought out process that I can validate those things in a way that starts to remove some of the challenges that can come along with AI software development, because guess what, it is still a large language model. It still going to get a lot of things wrong. It doesn't matter how much of prompting you try to give it. It is going to screw stuff up just like a human would.

How do you start to put in those checks and balances that remove that waste and be able to allow you to be able to move faster? We talked about that team size, if you are able to double, triple your coding output and you are trying to manage PRs and double-checking every one of those types of things, if you have 10 people working inside of a code-base and you're all submitting multiple PRs a day, how are you going to make that work? That is a new friction point for teams in general because you now have a significantly larger set of code to be committed and drove forward and all those kinds of things. There is a lot of waste that we haven't experienced in the past because we are moving faster than we ever have.

Ken: What's the path forward? How do people think about this stuff in such a way that they can address some of these things?

Chris: What I see a lot of currently is people are afraid to get started and they're afraid to get it wrong. If anybody sits here and tells you, including me, that we know how all of this stuff is going to work next year, the year after, they're lying to you. What you need to do is get in there and start to figure out how you can operate and get the most value out of these sets of tools.

How do you start to understand the constraints and understand how certain tools tend to do things? I've been using Cursor a lot here recently and it loves for whatever it's using Claude underneath, but however they have it put together, it loves to just randomly build fallbacks. It just does, and it's like, "What? Stop it." Understand how that works and the new problems that are going to come up because really the shift in value is going to be understanding your new process, understanding how to work together as a team with significant more code and being able to validate those changes and identify those bad patterns that are going to naturally come out.

Ken: One assertion I've heard, and this relates a little bit to the AI mirror thing we're talking about a little bit or as well, is that these tools and these processes will make good developers better. They know what good-- What it looks like and all those and bad developers worse. The idea of this is going to replace your juniors. First off, I guess before I assume that you agree to that, would you mostly agree with that, or?

Chris: I would agree with that, yes.

Ken: How do we get people from junior to senior now? How do they get the experience?

Chris: This is one of the bigger challenges of our day, of the work that I would normally give to someone else, a junior person, it's just easier to do it myself while I am doing something else. That's a serious challenge. I can have something running Cursor, going and doing something while I am on a meeting or while I'm doing something else and just casually keep an eye on it to make sure it's not doing anything that I wouldn't expect. It has raised the bar not only for junior developers, but really for--

To me, there's four kind of personas. There's coders, there's developers, there's software engineers, and then there's architects. Anybody that's in the coder, developer, it's a problem because you got hired because you knew how to write Java. You are going to have to now grow your set of skills to understand application architecture, understand the broader aspects of the work in which you're doing because I've heard this question a number of times now, should product managers be writing code? Those types of things.

Ken: Vibe coding anyone...?

Chris: It's easy to laugh at that in a lot of ways, and believe me, I have. There is a relatively serious group of people that are like, "Okay, well, I don't need to know the inner workings of Java. What I need to know is good software engineering patterns." It changes things a little bit.

If I was a junior developer right now, those are the things that I would be focused on of, "What is a good set of patterns and what is a bad set of patterns when it comes to application architecture," and not focus so much on understanding the syntax of a language or there's plenty of people that spend hours and hours and hours with things like lead code and understanding these really complex mathematical problems. Yes, all of that still matters, but having that broader architecture is something that we're going to have to focus in on.

Ken: That makes me wonder too, did you mention product managers writing code? Yes, we joke about vibe coding, but I'll be honest, I haven't written production code for many years, and the concept of vibe coding has helped me a lot in playing around with things lately. Does that imply the developers also need to learn more about the business? What is it that's going to make them successful as a business? It's the old joke of what does McDonald's do? If I'm a junior developer, and I'm looking at becoming more valuable to my company, is it architecture? Is it business? Is it both?

Chris: To me, it's both. It is always down to solving business challenges. That's it. We don't write software for any other reason other than to improve the quality of the business and being able to have a better understanding of product management, of the business, of the architecture, of why things are the way that they are and what you can do to solve them is going to have a huge impact on your ability to drive that value for an organization.

I think that is a very critical thing that people are going to have to worry about. It's going to push in both ways. Product managers that are more technical are going to probably be leaning into the software engineering, and then the software engineers need to lean in into product and product thinking and being able to tie that to build that business case, that business value is something that's going to be very important.

Ken: Shifting gears a little bit, another thing that DORA talks about is platform engineering. I know you talk about systems thinking. How do those relate? What is your take on that?

Chris: To me, you're only as good as the system in which you work in. I forget the exact stat, but it was like 95% of your efficiencies has to do with the system in which you work in. The developer tool chain really matters. What is the process that an engineer or an engineering team needs to go through to be able to get things done? I'm currently working on a client that it takes 74 tickets to get a new microservice up and running and into production. That's a problem.

What we're helping them with is this kind of systems thinking that says, "Okay, well, what are you trying to accomplish?" All of these rules and all of these tickets were put in for a reason. You don't want to look bad and go, "Wow, that was dumb." No, they were put in for a very good reason at one point, but the world has changed. What's the purpose behind it? It's usually some kind of governance, some kind of compliance, some kind of check.

How can you start to think about those things differently and say, "Well, maybe I can give you what you're asking for easily," and then have governance checks to say, "Hey, you're building an internal app that has a few hundred users. Maybe you don't need to throw that into a gigantic Kubernetes cluster. Maybe just drop it into ECS and move on." There's plenty of those kinds of things. To layer in the platform engineering part, you can make an argument that the software engineering teams themselves don't even need to worry about some of that stuff.

Great. Worry about doing your CD, get your image built, and then allow the platform to pick it up and move forward. That doesn't work for smaller organizations. It works for the biggest of the big, but that makes a lot of sense because you can now control the governance, the compliance, the cost, all of those things that tend to be huge problems when it comes to the individual software teams.

Ken: Something that showed up in the report, and I've heard a couple people saying, is this conflicting advice? Kind of related is the role of platform engineering did come out as very important, but then for-- I know this is audio, but I'm doing air quotes for listeners. The surprise ending was that the more user-centric the tools are, the better. Is that conflicting advice, or is that just how you're--

Chris: No. Honestly, I think those are the same. Your developers are your users, and you should be user-centric to your developers. What is the friction that they're having to deal with? What is the struggle for them to be able to move something forward? By thinking about them as your customer, you start to just change the way you think about things. When you think about it as a holistic system, you can then stop the local optimization for things. It's like, ''Oh, well, you just have to put in one ticket to make this happen.'' It's really not that bad, and fine. Well, but there's 73 left, and you're not thinking about it as a holistic system.

If you keep that developer lens on and say, ''Well, there's a thousand developers or however many, they're all having to do these things, how can I make this easy to give them that kind of paved path and optimize for laziness?'' We all want to cut corners. We all want to be able to make sure that I don't want to have to do three things if I can do at once.

By keeping that kind of user-like focus, you can optimize to make both of those true of optimizing for laziness and getting the best outcome, because those are the same thing, and that, to me, is the power of really good platform engineering and really good developer experience that comes along with that.

Ken: What are the techniques for understanding that? Are we talking about just user journeys? Are we talking about value stream maps? How do we understand what a user actually goes through, from I think John Willis says idea to cha-ching.

Chris: There's a bunch. There's qualitative survey data, there's quantitative system metrics like DORA, those types of things, but there's also different feedback loops that you can get from people. Early on in the AI tools, I thought it was very interesting when you did a quarterly survey and asking people how much time do you think you saved by leveraging AI tools?

That was very different than when you asked them immediately and doing like, ''Oh, hey, you just submitted a PR, how much time do you think you saved on the PR with your AI?'' and that was different than when you asked them beforehand, ''How much time do you think this will save you in the future?''

You need to be able to understand and gather that holistic set of data. Do interviews, do value stream mapping. That's where the power of these kind of developer experience group or an insights group, because you can start to pull all of that together and say, "Well, people think they're really effective. They're not as effective as they actually think they are, but this is why," and it's almost never exactly what you think it is. Having that kind of inquisitive point of view helps you build the systems and keep proving those systems through that developer tool chain to get to the outcomes that you hope for.

Ken: It's interesting, because I have a friend who's working on a very large codebase at an organization I won't name, and he joked that they need to bring the ping-pong tables back in, because in the 90s, it was waiting for the thing to build. The joke was for those of us that are gray, that you'd go play ping-pong while the thing was building. Now everything's faster, and we got all the AI, and he said, ''No, now I need a ping-pong table for when the AI agent is thinking.'' He mentioned a confirmation bias where, ''It only took me five minutes this time. The whole process still took considerably longer.'' Anyway, I--

Chris: That's right. I think that needs to be part of the workflow, and what I've found myself doing is a lot of times, I will create essentially an AI-generated plan for it to go do its thing. I'm going to work through it, have to break it down, understand back-end changes, front-end changes, like all of those things, get it really well grafted out and understand what that's going to look like.

Then I'm going to say, ''Hey, go do this thing. I'll check in along the way.'' Well, that now gives me time to go do something else. Whether it is, what is my next thing that I want to do, being able to just have a mental break. Like you said, go play ping-pong to do something where it's not just one to the next, to the next.

I like to take a joke at Scrum and say, '' Well, how many sprints can you do in a row before it becomes a marathon? '' Cognitive load is a serious thing, and being able to take that mental break when you need it while work is still being done, or focusing in on that next problem or that next challenge, or answering questions for others, doing some PR reviews, those types of things. All of that stuff can play together, and you can still be much more effective along the way.

Ken: We talk a lot, at least on this podcast, about AI as augmenting current people that, we mentioned 10 or 15 minutes ago, it's making accomplished developers better or whatever. How do we go from that, or do we to AI-first? Are we just going to sit on a beach and have AI write all our stuff? What's the path?

Chris: There's more than one path in my mind. One of the interesting things that I think we can get to pretty quickly is a zero-bug policy and saying, "Hey, I can create a set of agents and point them at my codebase and say, 'Go look for bugs and if you find one, fix it,' or having this intake of an error happens inside of my application, great, it gets logged and it gets dropped straight to an agent to have it take a look and try to figure it out and then spit out a PR to someone." One of the kind of-- I don't know if it would be a DORA-metric, it could be, but time to bug fix it.

I think that is a space that's ripe for that type of full automation because you don't really need it to add more, it's just take the context that's there, understand why something broke, and then fix it and then let me know what you did. I think in those types of scenarios, you can get to a relatively autonomous type, where the human's still in the loop, but it's less in the loop. When it comes to building new features, building new complex sets of things, that's where things can go horribly, horribly wrong.

Especially when you're talking about microservice environments, really broad, complex distributed systems, those types of things where you have your little AI tool looking at your little Java microservice, and it needs to understand this broader set of stuff and the implications of what it might be doing to be able to do what you want it to do better. That's going to take more of a human involved to be able to make that happen. It's never one-size-fits-all. As you're thinking about these different sets of problems, great. How do you start to define the teams and the people to be able to help with that?

To the question you asked earlier around how do junior developers start to understand more, "Hey, you're on bug watch duty for a while. Go watch this thing and see how we all screwed up and how it fixed it and understand what those patterns look like, then let's work into these other things." It's just going to be different. Again, none of us know what that looks like right now because things are changing incredibly quickly. There's definitely room for the people that are curious and that are truly wanting to do this.

Ken: That's what I was hitting at earlier. We joke, and we laugh about the bug watch thing, but that was my first gig. I got added to a thing, and I'll never forget it. The senior system administrator was like-- I came from the sysadmin side said, "Hey, if you start in a kitchen, you start by washing the dishes. Same thing here. You're cleaning up the dirt."

There's no better way to learn, but I worry if that's fully automated, how are those people going to learn those things? Host interjecting. I shouldn't do that. If I'm looking at using AI in that context, where it's watching my production system watching for bugs, my alert system goes to it, et cetera. Part of me, because like you, I think straddle the technical and business side, I'm seeing my bill go cha-ching, cha-ching, cha-ching, cha-ching, so there's a temptation then to use cheaper models, open source models, et cetera, but is that the same training data? Do you have any advice? Are we far enough along where you could give a CIO or a CTO advice on how do you balance the cost of the bleeding edge tools with the real world of paying the bills?

Chris: That's a really great question. I think this leaves us on a positive note for some of the younger among us and other people we are moving into, I think, the golden age of modernization and the building in these new systems that can allow you to do what you're talking about. There are plenty of these off-the-shelf tools that would love you to be using their most current, most up-to-date, smartest model to do everything under the sun for just a small transaction fee per API call.

That is the most expensive way to go about doing things, and there's absolutely things that can be done with some of the smaller models, the local models, or I think the future for a lot of especially bigger organizations are these SLMs or small language models that are specifically built around, "Okay, we're going to take that model and we're going to train on top of it, our context, and then be able to start to lay that in, in a much cheaper and easier way."

You need people to build all that infrastructure. You need people to build those systems. You need people to put all of that together in a really meaningful way. It's easy to look at the current market and go, "Oh, my gosh, it's doom and gloom for software developers." I actually don't think so. I think for the next foreseeable future, there is going to be more than enough work for the ones that are actually curious, for the people that went into software development because it paid well, and they just don't really care for it that much. It's just a thing. Those are going to be people that are going to really struggle. For us that just love and breathe and sleep technology, this, to me, I think, is the beginning of a golden era.

Ken: Interesting. I'm going to close on that because I think that is a nice, upbeat way to end it. Any final words, any takeaways? If you listen to the podcast, is there one thing you want people to learn?

Chris: Go download the DORA report. It really is very good. Go do that assessment of which archetype or persona are you? Understanding who you are and where you are is going to give you that insight into how can you go and make yourself better and driving forward. This is a wonderful time to be in software development and check out the DORA report. It's wonderful. We have a ton of other content on engineering excellence and our broader practice around this stuff. More than happy to share any other content.

Chris: Cool. Thank you very much for your time.

Ken: Thank you.