Rethinking the software development process for an era of specs and AI agents

Transcript

Ken Mugrage: Hello, everyone. Welcome to another edition of The Thoughtworks Technology Podcast. My name is Ken Mugrage. I am one of your local hosts, or one of your recurring hosts, I should say. I want to introduce a co-host today that those who are regular listeners have heard a little bit lately. We've had them as a guest on a couple of episodes, and we're pretty happy with knowledge and everything else, and so invited Caer Sanders to join us in the hosting panel. Caer will be joining us as a co-host for today. Caer, do you want to do a quick intro again, please?

Caer Sanders: Of course. Thank you, Ken. Hi, everyone. I'm Caer Sanders. I'm a principal product engineer at Thoughtworks, and I've been here for about a year and a half now. My background covers a range of things, from distributed computing to AI systems. Oddly enough, I've been building AI since before AI was in the mainstream, so it's been very interesting to watch it transform.

Ken: Great. Then for our guest today, it's interesting, at least to me, one of the things that I like the most about Thoughtworks-- I've been here. I wasn't gray when I got here, and I am now, but I've been here for a little while. One of the things that I always thought was really cool was the way that we, at least, try to build a culture where people are learning and testing boundaries. Then often, it's a double-edged sword, they go off and start their own thing.

We have a long history of that, all the way back to Selenium before I joined, to continuous delivery when I joined, microservices and data mesh, et cetera, and AI is no different. What we have here is a couple. They're former Thoughtworkers. I hope we can take at least a little bit of credit for their experience and expertise, but I'll let them introduce themselves. First off, Cassie.

Cassie Shum: Hi, everyone. My name is Cassie Shum. As Ken just mentioned, I was a Thoughtworker for about 12 years. Absolutely, Thoughtworks gets a lot of credit for a lot of the foundations and things that I've learned over the years that essentially has brought me to where I am today. I think after Thoughtworks, I joined a startup called Relational AI where we focused on a lot of knowledge graphs and the context layer, as a lot of people are talking about right now, which I believe is very applicable to the new AI agentic world that we see ourselves in. I'm super excited to be here. Thank you, Ken and Caer, for this invitation. I'm excited to chat about what AI means today.

Ken: Cool, and Tim.

Tim Cochran: Hi. Yes, my name is Tim. I also was at Thoughtworks for a very long time, for 19 years. I'd like to say that I grew up at Thoughtworks. I learned my craft through lots of pair programming and TDD'ing for many, many, many years. The last few years I was at Amazon, working on AI adoption and AI native transformation. In general, just a big interest in developer experience and Developer Effectiveness. That's what's somewhat driven to what I'm focused on now.

Ken: Cool. What we want to talk about today a little bit is the idea. We've covered this before, but I think Cassie and Tim have a little bit different take on it. How teams need to behave differently. We need to not just add AI, bolt it onto the end of our software development, but the actual way you create software is different. A term that we're seeing a lot these days is loops.

Thoughtworks has written a little bit about it. Cassie and Tim have talked a little about, and so forth. I guess, first off, we're saying, every team already has a product loop and supposedly an execution loop. What's all the hubbub about loops? What are we really talking about here?

Cassie: I think loops is our next-generation word for how do we close the feedback gap? How do we learn from what we're actually doing? When we talk about product in execution loops, they're very much the SDLC process or the PDLC process that we saw from years ago, all before AI. I think that's number one. We're bringing that back as a first-class concept. I do think, and I'm excited to hear Tim's opinion here as well, because we've been focused on both of these loops. Is that as engineers, we are seeing in the enterprises and companies right now that there's a major focus on the execution loop.

There's a lot of software factories and harnesses because the agentic technology that has come about in the last couple years has really accelerated that. I think the major focus of that has been accelerating. One of the things that Tim and I have discussed at great length, and we're trying to work with our customers on what that actually means, is it doesn't just stop from the spec to the PR.

How do you close that loop from a value sense? How does that actually come back to the product? That's why we talk about the two closed loops, and they're both equally important, and they need to feed off of each other.

Tim: Definitely agree. I think what's interesting is that they're optimized in completely different ways. I think from the beginning of software development, we've always wanted to industrialize software development. That's been difficult because most of the time we're actually creating something that's new and novel. I think now, especially with AI and some of the insights that we get from agent traces and things like that, we can be explicit about these two loops. Where, actually, you know what? We do actually want to create a software factory or a dark factory, and we do actually want to industrialize things that are concrete and deterministic. There's a lot of software development that is not concrete and deterministic.

Those things we actually need to acknowledge and accept that they are fuzzy, and they have to be discovered and evolved, and iterated on. Embracing that asymmetry between those two loops is where we see the team working on. Obviously, right now, there's a ton of focus on improving agency efficiency and creating harnesses. That's all great, all in the execution loop, but we also want to use the impetus of AI to actually close the gap between product and engineering. Because right now, developers don't have to spend two weeks to build a feature. They can actually come up for air [chuckles] after half a day or a day, and they can maybe collaborate a bit more. That's more about why we're thinking about these two loops.

Cassie: One last thing to add on this one as well, and Tim and I have discussed this a lot because we've been hearing this from actually some of our previous places that we've been is that, because now the agentic lifecycle is up in front and center and everyone says, "Let's bolt on the AI," there's such a focus on spec writing. How do we actually hone in that spec writing so we can hand it over to the agents?

As I do that, I feel like I'm saying, "How do we make things more waterfall? I'm going to upfront design specs, and then the agents will take care of all of that. I always ask people when we're talking to them, I say, "When have you ever had an idea that was that same idea at the very end when it's in front of your customer?" Half the time, if not more often time, they said, "Oh no, it's not," because, going back to some of the fundamentals that we've talked about for years, is you have to iterate and learn on that idea.

The question now begs, "How do you do that with the agents? How do you do that with builders and agents and iterate that and create a system to do that?" That's why those two loops are incredibly important as well. We don't want to be the new waterfall. We want to actually embrace that collaborative approach of actually creating an idea and finding value in that using the agents that we have now.

Ken: Is it the same problem framed differently? If we think about the DevOps movement the last 16, 17 years, that was all supposedly, and I say supposedly, because it got hijacked a little bit, about feedback loops. Making sure that product teams were seeing how things run in production and they could react faster and et cetera. Is this the same problem with different clothing, or is it really that different because it's AI involved?

Tim: I would say it's absolutely the same problem. [chuckles] What old is new and all that kind of thing. I think really what AI is doing is accelerating. The promise of agile or the dual-track software development, we've always wanted to have this close collaboration and this feedback. It's just embracing that just because of the acceleration of AI, I think.

Cassie: I think that acceleration is going to expose all of the different bottlenecks in the software development lifecycle process. It's not just engineering anymore, and I'm going to humblebrag Tim a little bit. Five years ago, he wrote about the Developer Effectiveness. He wrote on Martin's blog about how to actually hone that in and make that more effective. He talks about the bottlenecks that developers faced back then.

I think now we're in a new phase where new bottlenecks will come out because agents and AI are accelerating if you do it well, like accelerating the engineering area. The developers, the friction that was held and felt for many decades, and now we're going to find the friction and the bottlenecks in other areas as well. I think the premise of the feedback is the same, but the bottlenecks are going to appear in different areas. Tim?

Tim: I think it gives some examples. Some of this may come down to the fact that we have more data. How do we understand developer sentiment and friction? Historically, it was through surveys and pairing and things like that, but it's very difficult. A little bit with AI, and because we have traces, you can ask Claude. You can actually go to Claude and say, "Oh, the last two weeks, how did I coach? What was I doing to coach the agent?" You can get all this kind of insight that you couldn't have done previously.

I think a lot of times with developer experience, there was this hidden work that nobody knew about that the developers were doing and getting frustrated about, and was not very obvious. Examples might be the sort of hidden validation layer. You might have the most amazing test coverage, but did this software actually work? A lot of that was fairly manual. I think we're now actually through optimizing this execution loop and measuring the amount of manual coaching and interventions. We discover these weaknesses in our test pyramids and things like that.

Caer: I think hearing this, one of the questions I have as you talked a little bit about the difference between these execution loops and the product loops. The execution loop it sounds like is that spec-to-PR below, making code, generating it. What do you see as the product loop, the other side of that?

Cassie: That's a great question. I think some of the work that we're working on in terms of the product loop and bringing some of the agentic AI framework into that is really about two things. One is, I would say, bringing the context that, say, the engineering teams have originally had for a very long time, and bringing that closer to the product loop. So that the ideation, the things, the hypotheses, and the product feature thought process is actually grounded in the tech estate that your organization is grounded in.

I think one of the things that is very powerful now is that we can have business owners, product managers, people who have these really wonderful ideas, and developers to bring the ideas to bear with a shared context that you can bring in from architecture standards. How the different tasks are working out, the different repositories, and things like that. The reason why we can do that is we can pair with the agents who can actually siphon through all of that information very quickly and validate whether this hypothesis is doable, for instance. Or shape it in such a way that it's now removing the toil and churn that goes back and forth between product and engineering historically.

I remember back in the day, a product manager would say, "Hey, this is a great idea," then an engineer would be like, "Oh, you can't do that. Not in this ecosystem." I think even in the smallest sense, you can actually start collapsing that a bit in this agentic framework and really collaborate a lot more, given you can have a shared context with the agents.

Tim: A couple of interesting things. Part of what we're trying to do here is almost empower the team a little bit more. Because I think, intentionally or unintentionally, sometimes with development, you just have so much work to do that you don't have the capacity, even if you are empowered, to understand the customer or look at the product strategy, product metrics.

I think if we can bring the team together a little bit more, and because if the promise of AI holds true, developers have more time. Technologists have very good ideas, but they have to have the information. They have to understand the customer, those kinds of things. I think having these two loops can be very simple. Like having an idea registry or somewhere that you can have an idea and that product and developers can work on it, because at the moment, the constraint isn't so much about building it, it's actually, "Is that idea worthwhile? What will cost me? What's my evidence behind that idea? Why do I think it's useful?"

You did bring up an interesting point, maybe unintentionally, Caer, which was around the execution loop and the pull requests. This is a very interesting point, and because I think a lot of AI is measured on pull requests at the moment. With the execution loop, I think, it depends on the environment, but ideally, we try to measure until it's actually released software, I think.

I think it depends on the setup, right? In an ideal situation, everything shifted left, and your PR represents something that is releasable, but that is not the reality in a lot of companies. A lot of companies there's a lot of work that happens after that pull request. I think part of that execution loop is actually understanding what is happening after. Whether it's a manual. Ken, this is all DevOps, this is all classic DevOps, but if it's manual QA team working on it or whatever, slow end-to-end tests. Let's just move all that shift left.

Caer: It's funny that you mention DevOps, because I think a lot of the challenge of DevOps historically for me at least was more organizational. It wasn't even the technology; it wasn't anything; it was just how people were structured. Conway's Law in action. I wonder, with these different loops in AI, do you think it is an AI issue or do you think it's more of a rethinking how we work as teams?

Tim: I think it's more the latter. The rethinking. I guess what we're trying to be is, maybe we're just moving too fast into this next phase. We don't really quite understand what productivity is anymore. As a developer, how do I become more effective? Should I be measured on the amount of agents that I command, and my goal is to command larger and larger swarms of agents that produce more and more PRs? Or do we have this opportunity to, as you said, it's almost like using the impetus of AI.

I think everybody at the moment, lots of folks are open to change and rethinking the SDLC to actually really think about not necessarily just software development, but product development. How we're doing it end-to-end, how we're thinking about team structures, and how we're bringing people together? Absolutely, I think it's about the org transformation and those things.

Ken: A phrase I've seen, I believe it was in one of your writings, it may have been on your website for your new company, was the idea of a connective layer. I think that's where this differentiated, in my head at least, from all the other talk about loops. Was I hallucinating, or what is that?

Cassie: Maybe I can jump in here a little bit. I think the connective layer, or another frame for it is the context graph, or the layer where the two loops sit on. Goes back to actually very much organizational things that we have seen in the past. Now we have the opportunity to bring some of that organizational knowledge into these loops. I'll give an example of something that's happened before, and then what we can actually do now about it.

I find that before, as all of us engineers, product managers, we go in and we're like, "We need to build a product, right?" Where do we go first to understand an organization? We go into the wiki, we go into the Confluence page, we look at architecture diagrams, we go and talk to the expert on the ground who has the tribal knowledge, just to actually go into that repo and then go into this repo.

A lot of the things that we practice in terms of collaborative team energy in the organization to build good software is to gather that collective knowledge. We did it very ad hoc, and we had teams that knew certain things, and you'd Slack someone here and Slack someone there and things like that. We're not saying move away from that, but there are some things now where when we have agents and AI, that, as Tim said, is accelerating everything.

Imagine in an organization that an agent is now going into all of your wikis as an organization, and if your organization is 20 years old and you have a lot of knowledge in that wiki, how do you know what is the truth? That agent is not going to decipher what is true now versus what was true five weeks ago versus what was true two years ago. Now we get a lot of different types of knowledge that we can't make sense out of. How do we actually create software on all of that knowledge?

One of the things that we talk about is this connective tissue. This still takes collaboration with the teams to understand what is important in your organization. Is it these architectural standards? Is it these testing rules that we need? Is it these domain bits here that actually make sense? That's what we're moving forward with. It's also like what is in play right now? What stories are actually being developed now? What was blocked? All of these things.

I'm talking about this organizational knowledge within your software development team that can now be collected as a first-class citizen. Your agents also have that priority in terms of how to build these things. I think that's what we mean about connective tissue. That's just as important because of the context windows that these agents have and the lineage and all of these kinds of things.

Ken: How do we get people to actually participate in that? I'll be honest, I'm playing devil's advocate a little bit on purpose. We've had knowledge management-type stuff forever, and if nobody writes it up, then it doesn't get in there. Part of my job is reviewing blogs, technical writing and that sort of thing. People are out there doing AI-based searches, and they're finding marketing material and assuming that's from someone's real experience, and they can't wait for it all the way to, and I promise I'm not trying to be an AI naysayer, but this actually is true story.

For some reason that defies logic. We have two puppies under the age of 16 weeks in my house right now. I asked ChatGPT, I said, "How many weeks old is," his name's Mongo. "How many weeks old is Mongo?" Because I didn't want to count weeks, and that's what the vet wanted. It gave me an answer. I said, "Okay, and how many weeks is George?" That's the other one. He gave me an answer.

I'm like, "Wait a minute, Mongo's older than George. These are backwards. What the heck is--" "Oh, sorry." If yesterday the latest version out of insert vendor name here couldn't count weeks properly, can I really trust it to know which context matters?

Cassie: You can trust the answers that come from your agent based off of what you give it. That priority is that context layer that you, the SME, need to actually feed into the context. One of the things that I find a bit troubling with a lot-- and I fell into this pattern myself, because it's easy. It's like I'm talking to my agent. I'm like, "My agent knows a lot." You learn to trust, you learn to trust everything that comes from it, because it is grabbing from multiple sources of data that you can't even read fast enough.

I think one of the things that we've all seen in the past year or so is that we get a lot of AI slop, we get a lot of hallucinations, we get a lot of things because the data is not curated for that particular agent. The way we're thinking about it is to just turn it around and say, "How do you have specific agents that you're feeding the context that you care about so we can make the right decision?" Then, the biggest part that I know Tim cares about a lot is then how do you verify that those agents are coming up with those right answers based off of that context that you give it?

Tim: Yes, of course. It's a massive problem. It's always been this problem of documentation drift. We can't magically solve it with AI. but there's certain things that we can do with a lot of documentation. You can detect documentation drift for your legacy systems. I know there's lots of innovation around generating quality information for an existing system. I think what we're trying to do, or what we think the best practices are, is make it easy to bring in that relevant quality information.

One big thing is, when you have a feature or something, do we know the intent of it? It's not necessarily the requirements. Do we know the intent, and what do we hope to change when it's out there? Often, sometimes, at some point, somebody did that when they were imagining it, but it doesn't make it to the development team sometimes and gets lost along the way. I think you can create tools that allow this collaboration where those kinds of things, that lineage is actually captured and that helps the AI, that background, and will help the AI and the developer.

Caer: So many questions I want to ask. I think the big one I have is my team's right now; we're working on business intelligence systems. You can imagine intelligence systems means ingesting a lot of business intelligence to do that and validating it and making sure it's right. I wonder, considering there's this context there that you might be building to support these different loops. How are you thinking about supporting people bringing their business intelligence to the platform in a way that's scalable and doesn't require a fleet of forward-deployed engineers to do it for you?

Tim: I would say there's a lot of innovation in this space. I think there's a lot of tools that will help you clean and identify information. Some of this comes down to, how do I know if information is right and high quality enough, and also efficient? Because an agent can waste a lot of tokens just reading through reams and reams of things that didn't really care about.

I think it is relevant for certain types of information, like maybe architecture documents or coding standards and things. I think benchmarking is a useful activity. Especially when you're benchmarking your harness. A lot of what actually improves whether you can complete a task or not is the information that feeds into it. It's historical benchmarking. That's a combination of the context, your skills, your MCP tools, and your tests. I think that's something that can help.

Ken: Cool. Caer, do you want to ask any more follow-ups there?

Caer: One other one I had, and it might be too early in the conversation, so we'll see. We talk about context engineering and building this shared understanding of the product loop and the engineering loop. My mind goes straight to multiplayer AI and the whole issue of how do we have teams of people collaborating on these context datasets and with these agents?

Let's say you have an agent or a system that's AI-driven that two or more people are using and collaborating on simultaneously. It's not like it's just one person talking to a chatbot in a room by themselves, but it's more like three people talking to the same chatbot at the same time.

Cassie: I think this is actually quite key in terms, and we'll see what the future holds for this. I would say that having multiple people talk to one agent is one interesting example. I would say that having that shared context, it doesn't matter if that agent is the same or it's the next agent. If you have the source of truth as that context and then the agents are actually playing off of that one, then I think that would work in future.

I sort of have been thinking about your question in terms of what different roles some of these agents can play. These adversarial agents, or this analysis agent. What skills are we decorating our agents with in order to drive out some of these best practices or even some of these collaborative practices as well? I would say, in some of the systems that we're building right now, we have an analysis agent and a Spec Writer Agent that is about discoverability. Thinking about what the hypothesis looks like, and then working with yourself and myself together to come up with the best type of hypothesis to really drive out some of these acceptance criteria and stuff like that.

Then in another sense, I would say there's another agent that's decorated with, "I want to challenge. Is this actually going to bring value? Is this actually going to do this? Is that going to do that?" I think that tension, even between the agents and us humans, is needed in this collaborative sense to what Tim said earlier, is around "How do we bring the best value to our customers or in the software at the end? What is the actual intent here?"

What we're trying to think about is not agents replacing what we're doing, but enhancing that higher-level thinking that makes the software what it is. That's what we're here for. I think that's the way I want to change our thought process in terms of the way we collaborate and the software gets built.

Ken: One pattern I've seen that I find interesting, and we're seeing it ourselves too, is consultancies get out there and can see these things and recognize them. Sometimes correctly and sometimes not, and are rarely positioned to offer products to the space. You both talked about multi-years at Thoughtworks, and we used to have products, heck, I used to work for that division, but mostly as consultants. What led you to believe, whether it's Relational AI years ago or your current one that you're working on getting off the ground? Where are products better able to solve these than a consultancy? Or what led you to the product offering, I guess?

Cassie: I'll give you my opinion of why I'm excited about building a product is actually something that I will put Tim and I on the board, I would say maybe 9, 10 years ago. We've been friends for a very long time. I remember over dinner one night we said, "Gosh, we keep talking about the similar problems that he and I were both facing, but we were on different customers, and we kept seeing the same patterns,” and things like that.

We said for a long time, "My gosh, wouldn't it be really great to have these accelerators, to have these bootstrap things?" These patterns are known. Of course, there's some bespoke problems here and there, but at the general sense, we can identify those patterns. We've grown up in this world where we became very good at pattern matching. I think at least what brought me back to working with Tim again is that I remember that conversation.

Now, in this world of acceleration, and the two of us being able to accelerate putting these patterns that we already know together using agents was what brought us around to, "I think these products can now accelerate this." I wouldn't over-index on this product as the silver bullet, no. I would say that these are the things that are going to accelerate AI impact for companies. That's how we should think about it.

Ken: I couldn't agree more. Every customer is like, "Oh, but our problem is that's unique." It's like, "Well, [laughs] not so much." How have your conversations been? Are people like, "Oh, yes, you're right. We get that there's something called the 80/20 rule. Call it whatever you want." That there's a lot of overlap here, or is there pushback? What's the response been like?

Tim: I would turn it around more about the problem. I think most of the customers have the problem, which is more around the, at a broad level, it's not seeing the AI impact they expected. At more granular levels, it is. A lot of it's about the upskilling, the changing of habits day to day, and how developers are dealing with this new world. I think a lot of that is common.

Ken: What should those of us who still live in the consultancy world, which is a large percentage of our listeners, if I'm honest? What are things that we could change or should be changing about our behavior to get people to recognize this more?

Cassie: For me, I would say, as a practitioner, lean in hard with everything that's going in with these agents. Not to say that you need to be dependent on all of these things, it's really to create empathy for the folks that you are consulting with. I will personally say that I called myself post-technical for years because I was. [laughs] I was the architect who talked about design and patterns and things like that.

I've been very fortunate to have the opportunity to go back and actually code with these agents and understand what their limitations are. How you can actually use them as opportunistic enablers and things like that. I wasn't really truly able to conceptualize that until I was on the ground building with the agents and seeing what they can and can't do. Then it's a flywheel because everything's changing.

To stay in that mode right now, I think it's incredibly important as a practitioner because that'll just make you so much more powerful for your customers, who, frankly, a lot of them are very scared. That's what I'm hearing. There's some fear from the engineers, the practitioners on the ground are, "Is this going to take my job?" The answer is, get in there and see what you can actually use them as an enabler versus as a replacer.

Caer: I really love the note about having empathy for consultants, users, for example, our customers. Because, ironically, I was something of an AI skeptic for a while, and in some ways I still am, but the one thing I do is use all the models as much as I possibly can. Only because using them shows us how good or bad they can be and gives us a real understanding for it instead of just hyperbole.

One of the things I heard earlier, you were talking about silver bullets. I was thinking to myself, I like it when a product describes itself as not a silver bullet because it knows what you are best for. I'm curious, from where you are right now, what do you think is the thing that you're going to be best for with this new concept of the different loops and bringing them together with a context layer?

Cassie: I think it's about to bring the humans back into the loop in a productive way. I think that's the differentiator here. We're not here to accelerate and get rid of everything. We're here to actually utilize humans in the way that we've been wanting to utilize in a long time. That's what I think.

Tim: It's easy to say, but focus more on creating better products, I guess, [chuckles] rather than just more products, better products. I think that's what we're going to do.

Ken: Sometimes I wish this was a video blog and not an audio blog so the listeners could have seen Caer and I applauding for bringing the humans back in. Appreciate that. The question I like to repeat to every guest is that a lot of this is in formative stages, and I get that, but is there anything you'd give our listeners advice you'd give them Monday morning, when they come to work? They're mostly practitioners. What should they be doing? What's an actionable takeaway? Is it go read a book? Is it go do a thing? Is it try another model? What should they do Monday morning? I'll go one by one. I started with Cassie, so this time we'll start with Tim.

Tim: I think what I would probably say is slow down. At the moment, there's so much pressure to be producing more and more and more. Especially when we think about being effective. I think with AI, you get bombarded, and you're playing whack-a-mole with all the agents and blocked and blocked and blocked. It's too easy to get too much into that instead of thinking about, "If we come back to the execution loop, how are we actually improving our software factory execution so that it is easier to work with."

Then the other pieces, being more effective, maybe actually see how you can close the gap to your product, business stakeholders, and think about, like, "Well, can we actually create, rather than just more software, the better, more effective software.

Ken: Perfect. That gives Cassie the last word.

Cassie: Man, that's a hard one to follow. He stole my answer. [chuckles] I would say, very similar to Tim, what problem are you trying to solve? Instead of just diving into Monday morning and spinning up all those agents and hearing that sound that now is just forever in my head. [laughs] I would say talk to your teams, try to figure out what problem are you trying to solve today, tomorrow. Maybe always take a step back and say, "Am I still solving that problem? How can I better leverage the tools that we have right now instead of just getting in the mode?"

Ken: Great. Thank you very much for your time, Cassie and Tim. Caer, welcome to the jungle of being a co-host for our podcasts, and congratulations. Thanks for the listeners. Everybody, have a great day.

Cassie: Thank you.

Tim: Thank you.