Enable javascript in your browser for better experience. Need to know to enable it? Go here.

Developing an agentic-ready data strategy: A question of trust

Podcast host Nic Smythe | Podcast guest Amy Raygada and Jade Forsberg
November 19, 2026 | 57 min 21 sec

Listen on these platforms

Brief summary

Autonomous AI agents don’t solve data debt - they amplify it.

In this episode of Pragmatism in Practice, host Nic Smythe is joined by Thoughtworks data and AI strategists Amy Raygada and Jade Forsberg to unpack why unreliable data, not technology, stalls enterprise AI.

They explore how semantic gaps trigger costly hallucinations, why governance acts as the "brakes" that let teams move faster and how leaders can combine trusted data foundations with workforce redesign to scale agentic AI safely.

This conversation is based on Thoughtworks Perspectives Edition 39, "A question of Trust: Developing an agentic-ready data strategy."

Transcript


[00:00:00] Nic Smythe: Hello and welcome to Pragmatism in Practice, the Thoughtworks podcast that explores practical approaches to becoming a modern digital business. For the last two years, enterprise AI has been about prompts, large language models, and quick productivity wins. Now leaders are looking at AI agents that act on their own, making decisions and running workflows without a person approving every step. The thing holding most organizations back isn't the models; it's whether they can trust the data those agents are acting on.
 

By agentic-ready, we mean an organization where an agent can find the right data, understand what it means, and act on it within clear limits that someone is accountable for. Today I'm your host, my name is Nic Smythe. I'm the Head of Service Lines for Thoughtworks here in Asia-Pacific. Joining me are two of our principal data and AI strategists globally. Amy Raygada joins us, Head of Data & AI for Central Europe and Author of Data Products: Volume 1, and Jade Forsberg, Principal Data & AI Strategist who leads our Data & AI Modernization practice here in APAC. Welcome to you both.


[00:01:08] Amy Raygada: Thank you very much, Nic.


[00:01:09] Nic: Amy, we might start with you. Would you like to give us a brief intro and tell us a bit about some of the work you've been doing?


[00:01:16] Amy: Yes, so I've been working for 20 years in IT. I started in software engineering, and then I moved to data back in 2012 when there was not even a data engineer, and it was more like a DBA. Then pretty much I evolve with data as today, working more on the data strategy, AI strategy work. It's been very, very interesting, quite a ride, I will say, to see how data has evolved over the years, coming from something that was not that important, or it was always important, but until something that's now revolutionizing the entire way we live today.


[00:01:53] Nic: Jade, if you'd like to tell us a bit about the work you've done and introduce yourself.


[00:01:58] Jade Forsberg: Yes, so I guess similar to Amy, I also started as a software engineer and then moved towards data engineering, now data strategy. Thankfully, I was able to skip over the period of being a DBA and didn't have to get into those depths, and I went straight into data engineering. I've worked a lot with clients on how do you bring in the data as a product thinking, and actually implement data products and data mesh. Started with clients in London, then had the pleasure of doing the same in Singapore, and now here in Australia.


One of the big focus areas I have with clients is always tackling the age-old question of, "I've spent all this money on my data platform, but I'm not seeing the ROI. How do I see the ROI?" Just working through that. Now that same question is coming up with clients that I work with on: "I'm spending all this money on tokens and AI. I'm not seeing the value relative to the amount I'm spending. How do I actually start to realize that?" Yes, that's me.


[00:03:04] Nic: I think that leads well into our first topic of conversation. Just a bit of background for myself. I came through UI engineering into product management, so I have no background in data whatsoever. I'm going to be relying on you both to educate us in that respect today, but you just touched on a good point there, Jade. We're basing our conversation today on the Thoughtworks Perspectives Edition 39, "A question of Trust: Developing an agentic-ready data strategy."


One of the key topics we want to kick off on is probably why so many AI initiatives get stuck. Some statistics called out in that report are that 93% of IT leaders, as sampled by Thoughtworks and WIRED, are planning to deploy agents this year, but only 5% of IT leaders actually feel confident that their data can support those initiatives. Companies are buying the Ferrari and then finding no one's able to drive it because they have no roads to put it on and no one trusts what it's going to do. Amy, you talk about proof of concept symmetry. What do the projects buried there have in common?


[00:04:25] Amy: Actually, this is a quote from my book, "The Ferrari nobody drives." It's the reality that we see. I think from Jade's intro as well: in every client, they invest tons of money in a platform without thinking if your users are going to be part of it, if they are going to really use that, if it's discoverable, if it's usable in general, and then they are complaining why nobody's using this because they buy or they create these just for technical perfection, let's say it this way. I see this a lot with Agentic AI.


What I usually see with my clients is that they have 100 POCs, 200 POCs. Everybody's using AI, everybody's spending tokens every day, but none of these AI agents were created doing or using product thinking. When we see a use case, we're like, "Oh, I think that will be beneficial for my sales team. I'm going to create one." Actually, it's not how the workflow works. It's not what is going to actually create more effectiveness in their work day-to-day. It's actually something that they thought will be interesting.


They finish it. They put it into the POC. They show it into a board meeting just to impress. That doesn't go from there because no one is going to use it. When they put it into production, if they make it to production, usually it's too expensive to be there into production for a long time. They basically retire that in the first few months because of usage or because of cost. That's how the cemetery of POC is one of the most common things that I see.


The other time I was with a client, and he was like, "Hey, Amy, can you help me because we have this cool POC. We want to roll this off in 250 stores, but it's only in one store where we are doing right now the pilot is going to cost around €90,000 a year. What can we do to make it better?" I was like, "Do you really need this, because what you are doing right now could have been done with a traditional machine learning model instead of going with agents first of all, and have you used your user research to understand if this is what your users on the store is going to do, like is going to actually use it?"


He was like, "Well, no, but we thought that would be revolutionary because we wanted to have the stamp in our website and our stores that said powered by AI or something like that." This is what I see most of the times, I will say.


[00:07:00] Nic: You touched on a really interesting phenomena there. We are frequently seeing these circumstances where leaders, senior leaders, even ones that are fairly technically savvy, have been able to fire up their own proofs of concept, been able to vibe code something or get a copilot or assistants to help. Jade, how does the risk profile change when we move from copilots or assistants vibe-coding the proofs of concept or proofs of value that we might spin up on our own machines as leaders using AI, and then take them into agents that act on behalf of whole parts of our organization?


[00:07:41] Jade: That's a great question. When you're looking at how that risk shifts from an assistive copilot to an autonomous agent, there's a few things to consider. One of them is that even with an autonomous agent, it's a scale of autonomy that you might be implementing, everything from a single agent that's doing a very particular task through to multi-agent workflows where we're talking about multiple decisions and actions being taken. In terms of that risk profile, obviously the more decisions, the more actions being taken, the more chances there are of something going wrong and not being caught in that process potentially.


The biggest shift going from an assistive copilot type workflow to an autonomous agent is that you're going from augmenting and supporting a human who's still in the driver's seat, the decision seat, through to now having an automated system making that decision or taking that action. The key thing there is to actually think through, and I know people don't always like to think about the negative things, but you need to go through what are the potential failure modes of the agent that I'm implementing?


Regardless of whether it's a single agent or it's a multi-agent workflow, what are some of the things that could go wrong? We often do the security threat modeling as we iterate through delivery. It's that same idea, but now, thinking through failure modes of that agent or the workflow, and as we link it back to risk, you're starting to think through, in those failure modes, what does it mean for accountability? If you've got a human in the loop, they've got some level of an emotional incentive, either through job preservation or wanting the company to be successful. There is an incentive there to, in most cases, do the right thing and do a good thing, whereas an agent might not really have that.


The other thing is in that same vein, it's easy to hold a human accountable through various mechanisms. How do you now hold an agent, which is just a software system, to some degree, accountable for those actions? When you're thinking through that, the factors that come in is: well, what's the sensitivity, not just of the data that's being acted upon, but also the sensitivity of decision-making, because not all decisions carry the same weight, the same consequences, the same impact.


That's what ties it back to the accountability, but also that change from a human doing it to an agent doing it. Again, it all comes back to those failure modes of what could go wrong, how bad is it if the thing goes wrong, but applying that not just to the decisions, but the actions that the agent takes, so that sensitivity.


Then the second thing I'd say is the reversibility of that action or that decision. What's the actual cost of reversing? There, you're thinking about what's your reputational risk associated with those different failure modes. What's your regulatory and compliance risks? Are there fines that you're potentially going to be liable for? Then the actual cost of reversing systems. What does that mean for your database rollback, for your operational system rollback?


I think it's not that the actual risks change, but it's more how those risks manifest and what actually triggers that risk becoming a reality that is now shifting. Where it's no longer a human making a mistake or the fact that we didn't catch a regression in a software system; it's now we actually have an autonomous agent in the loop who isn't bound by the same incentives and controls as a human is in all cases. Those are just a few of the things to think about as you're moving from that copilot stage to those autonomous agents. Again, it's the sensitivity of the data and the decision, the reversibility of the action of the decision, and how do you maintain that accountability to manage that risk longer term?


[00:12:10] Nic: I think we'll get to guardrails and things like that a little bit later, and those interesting things. Might jump in, you're really talking there around the human in the loop, and then the autonomy that we look at these agents to have. Part of that relies on these agents working with the same amount of context and the same amount of tribal knowledge that a human in that loop or a person in that organization might have.


I'll throw this one to you, Amy. When employees are falling back to trusting their colleagues over their copilots, what's that telling us about the trust that they have in the data?


[00:12:54] Amy: That's a very interesting question. Actually, I was with a client last year, and they wanted to start a whole agentic journey. I went there with a product strategist person, and we were doing this assessment. When we were talking to them, basically, they said, like, "Okay, we're putting this chatbot there with some questions, but the people is not trusting and they don't want to use it. Why is that?" I'm like, "Okay, so have you done--" they have done, actually, a massive amount of user research inside the company because they sell data, actually, so they were asking many, many clients.


We went to some of the interviews as well to understand, and most of them they were saying that they don't trust that because they don't know what calculations are going behind. If they have a Monday morning dashboard, and they see that, like, "Okay, where is this data coming from? How am I able to prove that this is the right data? I don't want to use it. I'd rather, again, to download some Excel sheets somewhere, try to export the data," do their own calculations, and move from there.


They have different type of users. Some people older, some people younger, some people in middle age, let's say, but usually, if they have no way to explain- this is also part of what Jade said- the trust and also explainability of these, they cannot trust, so they go back to the old ways of working.


For example, I have seen that some people, when they are using their agents, they rather go and also say, "Hey, look, I never trusted my data team on the first place because there was always five definitions of revenue, five definitions of leads, and I didn't know which one to pick, so how or why would I trust right now what the agent is telling me?"


On top of that, it was quite interesting. A couple of weeks ago, I was speaking on Helsinki Data Week, and I spoke about one study from Cloudera with the Harvard Business Review, where they asked some leaders how much they think if the data is ready for AI. 7% said, "Yes, we are ready." Hence, 47% decided to still invest in AI because they think agents will solve their data quality problems, which really makes no sense.


How an agent will resolve your data quality problems where you never invested on a semantic layer, on context layer, or in general, with your team to understand what are the business needs, how can you become with a definition of revenue that goes across the whole organization or do naming convention and so on?


That's one of the reasons most of the people start, "Yes, this is interesting. Let's use a little bit of AI here and there," but then they are like, "Can I really trust this data? Is it really giving me what I need, or do I need to-- maybe I trust it, but I still spent a lot of time proving that this is right." I think it's also the way: when we involve the users into the whole process of identifying a process, they feel more trust because they were heard about what exactly is going to be created in the future, how it's going to be shaped, so they don't have a doubt on what is exactly the agent doing nowadays.


[00:16:25] Nic: You touched on a really interesting point there. I think in the move towards setting up autonomous agents and treating data differently in organizations, we've just moved that lack of trust from a data team that we didn't quite know or quite understand to a data box or a data platform that we don't quite know or understand. I think it's revealed a lot about the data literacy of a lot of business leaders.


Part of that is this break and this disconnect between operational data flows, the data that runs the business, the analytical data flows, the data that helps you analyze a business. Jade, where does that data flow typically break between the operational and analytical planes?


[00:17:13] Jade: When you look at the operational and analytical planes, typically the operational plane is your producers of the data. It's the actual systems that are capturing the facts that later feed into the analytical plane, where we can look at historical trends and analysis and use that as a way to then feed back the next set of bets or features that we want to build in the operational plane, which produce new data to come back into the analytical.


When you look at that flow, I don't think it will surprise people to know that it's usually not at the technical level where that breaks. It's exactly what Amy is getting at with the explainability and the breakdown of trust. Where that starts is usually with the actual ownership and the operating model and the way that people and teams are organized between those two planes.


The reality is that the tighter you can make those workflows, those teams, the more collaboration there is between the operational and the analytical, the better you have that flow and that cycle of sourcing data, learning, making decisions, improving products, learning if that works, making the next improvements, and so on. Often that breakdown comes from the ownership. Where you'll see in many enterprises still today, it's usually a single team that's responsible for the entire enterprise.


You talk about agents not having context and meaning on the data. The data team doesn't either in many cases because they're a central team having to deal with the requests of the entire enterprise. That's multiple different business domains, multiple different contexts. It becomes a more transactional: ingest the data, but it's disconnected to how the end user is actually going to use it.


That starts to erode the trust at each stage because you're not building and designing those data products, those facts and insights you need to serve to the business based on what they actually need because you're disconnected from how they're going to use it. That means that you omit things, or you don't model it quite right, or the freshness isn't quite right.


Over time, that erodes the trust because you're trying to serve so many different needs from the same central team that doesn't have that context, or the operational producing system changes and the data team doesn't know about it, and then pipelines break. Depending on how well your observability is factored in the platform, maybe they break silently. Now someone in the business hasn't had a refresh of data for several weeks, or maybe it just breaks all the time, and the team is overloaded in trying to respond to it. It comes back to making sure that the teams have the context and that they're working effectively across that data supply chain. That lack of context hurts the humans as much as it does the agents to a degree. It's making sure you have that accountability and ownership for the data.


The other thing I see happen that starts to erode some of those foundations is-- I'm a big fan of domain-driven design. One thing that we talk about there is the idea of bounded context, or areas within which a language, concepts, and terms are consistent. For example, customer is often an overloaded term that means a different thing in different contexts. Could be a lead, a prospect, an applicant. There are different permutations of a customer along that journey.


What often happens is that language, and that subtlety, doesn't make it into the data plane, so you don't have your anti-corruption layers in the domain-driven design terms where you're doing that language translation between each step so that you keep that consistency and you're not using those generic terms as much.


Where I see that break the most is when we take data from a producer, especially some larger ERP systems, like an SAP, where we just ingest the data almost as it is, and so you end up with a lot of technical names in your data. I've seen tables with column names that are like EX572-A. Forget giving that to an agent. No human is going to look at that and go, "Oh yes, that's customer ID."


Treating the way that we manage and handle that data with the same level of contextualization and anti-corruption in terms of looking after the language and the concepts at the data layer as much as in the organizational layer is another thing that typically breaks and starts to erode that trust over time.


[00:22:21] Nic: You've touched on something that is called out fairly strongly in the report. Amy, I might throw to you. This is what we would call a semantic layer or an ontology. What kind of context does that give an agent that they won't get otherwise from a data warehouse, and how important is that to solving these problems?


[00:22:48] Amy: This is really tied to what Jade basically said in terms of if you have customer and you have a customer as a lead, as a prospect, and so on, the agent won't be able to know which one to pick depending on the process. I have seen that plenty of times, so I chuckle when you said the column XYZW_ blah, blah, blah, because this is very common. How the agent will know this is the right one to pick?


Let's go through an example. We have Maria, the analyst. Maria is an analyst doing things manually. She sees an odd number, and she checks before she publish it. She sees maybe on the revenue entity book something change, but she's like, "I'm going to ask Robert because Robert works in finance, so I can go and talk to him, and he will know what exactly is going on." Robert will tell her, "Oh yes, we deprecated from the production system one of the fields of the table that was being used for this calculation," but actually no one knew about it.


Then everything just breaks. That's why the number, it looks different. If your observability, as also Jade said, is not there, that broke silently, the numbers just look odd. If you are not working with this data before, you wouldn't notice that there is a change, but actually Maria noticed those changes. Also, someone needed to give this communication about this field was deprecated in March, but no one had the ownership to give that communication, so that's why nobody knew.


Anyway, you have there another four definitions of revenue, for example. You will say, "Maybe should I choose this other one? Is this maybe the right one?" If you are not Maria, you probably will pick another one that the number is closer to what you think might be the number.


I was working in a company a few years ago, and I remember that we were creating a report for the board members. When we were seeing the presentation that someone put together, actually, the metric of revenue was different completely to whatever the data team was using and the monetization team was using. We're like, "Where are they even getting this number?"


Apparently they were getting these from some finance system that we had not even access, because they didn't want to share that data but also, we didn't have the contact with these stakeholders because they refused to talk to the data team because they have no trust in us because of previous situations that there were in the past. This is the context, again, that the agents don't have.


Maria can go and talk to Robert, can go and talk to many other people, can understand why these numbers are not making sense. For an agent, revenue is revenue. I'll pick up any of those five metrics in there, and then I will provide the data. That's why this is one of the biggest problems we're having right now. I have seen, at least in my region in Central Europe or in general in Europe, the rise of the data mesh.


Because now people needs to go back again to semantic layers, to data quality, to ownership, to governance, to all the things that we have been preaching for years that are never been taken into consideration, or they are too expensive to invest. They maybe did data mesh a certain way with a platform, and that's all. They never went through the operating model to the change management, to the ownership, to all the governance parts that needed to be taken into consideration. That's why agents are failing now, and they are coming back today.


[00:26:28] Nic: I'm really glad you touched on data mesh there. Shout out to our wonderful erstwhile colleague, Zhamak Dehghani, who coined that term in 2019. You speak there on a couple of topics. I'd like to move on to data products, accountability, guardrails, those sorts of things. I think people hear those terms, and it makes it sound like you're slowing things down rather than speeding things up.

I'm keen to understand how do you-- I might throw this to you, Amy. How do you build the business case for something like the semantic layer where there's no
immediate upfront return on investment? How do you make the case for those guardrails for the rigor of data products?


[00:27:22] Amy: I'm going to be honest with you. I stopped trying to sell the semantic layer on its own because usually it never lands. Because nobody sees a return of investment there, nobody sees the value, even though they are going to have problems in the future, especially with agentic. What I do instead is attach it to something that the business already cares about. Right now is AI. That's the hype of the moment.


Let's go back to the revenue or the customer example. If you put an agent on top of that data where revenue means three different things or customer means five different things, it will confidently give you the wrong answer. I can say at least 90% of the time you're going to get the wrong answer. The semantic layer is what makes the AI investment trustworthy for them and to be valuable for them.


Then I make the cost of doing nothing visible. The hours are spent reconciling all this data before every board meeting, before the decisions are made. This basically is stalled because two teams bring different figures, and they need to understand what exactly is happening there. I start small. Also, because of the product thinking of the data product. It's like, "Okay, what is the business value you want to get out of it? What is the top five use cases that you would like to have in there? Let's build a data product in there, but let's do it the right way."


Let's start understanding the problem, or what problem you're trying to solve here. Then let's go and see what data that we have available. What is the quality of the data? What kind of things we need to put on top of that? Let's build up on the data product. Let's create the right definitions of those metrics that you would like to put in your data product. Let's talk to business. Let's talk to the tech people. Let's talk all together. We start with this domain and with this metric.

 

Of course, this is not from today to tomorrow, because probably there will be five teams arguing on what is a definition for revenue or what is revenue without returns and how we are going to name it and so on. This is the hardest part. Then when they finish that exercise with the first, let's say, pilot POC or use case, they're going to get immediately with the AI agent the right answer. That's going to build the trust that they need to continue investing on this.


It's a trick, let's say, because this is the way you explain them, or you can show them, "Okay, if you want to go to the POC, let's go to the POC of your agent, but let's look at your metrics and which one is the correct." When they start getting all the time this bad number, that motivates them to work on something that will get them a real number that people is going to use. Also, for some clients, for example, is external reputation as well. You cannot go with an agent that is going to give you wrong numbers if you are selling your data, for example.


Internally, of course, that's going to be a problem, but at least it's contained within your organization. Reputation is quite important, especially nowadays with agentic, because a lot of companies are suffering because of that as well.


[00:30:34] Nick: You touched there, Jade; I might ask you about the cycle of intelligence. Our report refers to, and you've touched on it there, Amy, our report speaks to the cycle of intelligence where, to your point, value starts with a business decision, which then shapes the data supply chain, and data trust, data utility, data governance reinforce each other. Jade, how does that work in practice, and where does it usually break?


[00:31:05] Jade: I'll just build on what Amy was saying with, we don't really talk about semantic layers as much as the end thing to do it. It's really about the outcome. That's where the cycle of intelligence comes into the picture. When you look at data in particular, the value of data comes from how it helps you make decisions. Unlike delivering software, where you have a feature that you can actually touch and use and feel virtually, it's the decisions, it's the insights that data gives you that provides value.


Really getting ROI from data is making sure that you're able to make better decisions, faster, and have more confidence in those decisions. The cycle of intelligence really just encapsulates that, as you were saying, Nic, into the data supply chain. In practice, how that works is quite simply: for each use case or value hypothesis that you have, you start at the end. Start with the outcome and work backwards. You start with what actions do I actually need to take to drive the growth that I'm targeting, to drive the cost cutting that I'm trying to do? What are those actions? How am I going to know if that's working? That's step one.


Then you work through, like, okay, to take those actions, what decisions do I need to make? What's going to give me confidence in that decision? What's going to give me that trust that if I take this call versus that call, we can go ahead? That's where now you work backwards into what's that supply chain to get you that insight so that you can make that decision, you can take that action with confidence?


That's where then you get into, like, "Okay, now I know what I'm trying to influence. I know what decisions I need to make. I know what data I'd like to have to make that decision." Now you just work back to, like, okay, "Well, what things do I need to combine or aggregate or model to create that insight? What data do I need to source, and from what systems, to have that?" Then, going back to the original question as well of how will you know if that's working? What additional data do I need to source to track that?


That whole process becomes your cycle of intelligence. The faster you can iterate through that, the better you can make decisions; the faster you can make decisions, the more confidently you can make decisions. To give an example, we had a client who creates engines and other big machinery. They wanted to look at how they could optimize their cost through the amount of spare parts that they would order and keep in stock because they had large amounts of spare parts in the realm of multi-millions of dollars worth of spare parts that are sitting around, and if those don't get used, that's money that's gone to waste that you didn't need to order that many parts.


We worked through that and then worked backwards to, like, okay, well, to be able to decide how many parts to order, you need to be able to model maybe the remaining useful life of parts that are in existence. Look at how do you feed that back into and combine with what you already have in stock, what you're likely to need over the next six months, the next one year. We encapsulated that into governed data products that source data from SAP, from some of their other internal systems that were managing the inventory and the forecasting, and just connected those dots better so that they could make that decision.


That's really how the cycle of intelligence works and how it helps make sure that everything you're doing is grounded in, "What's the outcome I'm trying to influence? How do I make sure that everything I do is focused on that outcome?" That applies equally whether it's a data initiative or an AI initiative. It's just what changes now is, is the human making a decision, or is an agent making a decision?


If it's an agent making a decision, then when you get into that supply chain, that naturally leads you down to, "Well, how do I make sure the agent has the right context and knows how to assign meaning to the data?" To Amy's point, which definition do I use? Which column? Which table? How do I actually treat that? What relationships does it have with other concepts? That naturally brings you now to, "Okay, I need some form of--" whether that's a traditional BI semantic layer or the broader definition now of the ontologies and the taxonomies, or even just as simple as giving it access to your business glossary.


That cycle intelligence just keeps you very grounded in, "What's the thing I'm trying to achieve? What's everything that I need to achieve that?" You're just laser-focused on build what I need instead of building an entire vast library of data products that no one's going to use, or ingesting all of the data that nobody's going to use. Where that breaks is in that of the disconnect between the outcome, the actual data, and the teams that are running it.


The number of handoffs between teams, the fact that the team actually sourcing the data is disconnected from the action that's going to be taken, doesn't actually know how the data is going to be used. It's a lot of the same things we were talking about earlier with the flow between the operational and analytical planes, because the cycle intelligence aims to help some of that.


It's those disconnects between the teams where the team sourcing, modeling the data, doesn't know how it's going to be used, who's going to use it, how much more context do they need to be able to use it effectively. Or it's that by the time data and insights, because of those disconnects, are surfaced, or the quality isn't quite right, and so now that starts to erode the trust, and so that starts to again create more friction.


Ultimately, that means that then your data platform or your AI platform remains disconnected from the value that it's unlocking and enabling, because it becomes so far removed from the outcome when you have all these silos and handoffs. You're basically putting brakes at every step of that cycle instead of making it a streamlined highway with the automated tollgates to make sure that we govern it effectively throughout the process.


[00:37:56] Nic: You touched on a really good point there. From the recent Thoughtworks global CIO survey, 88% of CIOs say that AI adoption is outpacing their governance structures. How do we take that governance from being reactive gates, how do we stop it feeling like something that's slowing us down, to making those continuous guardrails that can enable teams to continue to move at speed, but give us the confidence in our outputs and our work?


[00:38:27] Amy: Well, I'm a fan of data governance. I also work quite a bit on data governance. I also understood over the years that following, for example, the whole DAMA wheel might slow down quite a bit your journey and might not be adopted as you would like. I'm a fan of governance that enables, not that constrains. What I mean by that is that we can actually automate quite a lot of the things in the Cloud. Like security, data quality, and so on, but also you need to have certain guardrails, of course, that you have to redesign with your team.


First, you need to go with your users, with the people who is part of the governance, have discussions, agreeing to certain level of governance. Of course, for example, in data mesh, you have different domains and so on. You need to have at least a set of guardrails that are the base for everyone.


This should be applied for every single domain in a certain way. Some domains really require more privacy, more security, more control, and they can design their own policies inside. You can do a lot of policy as a code, for example, to avoid having all this manual work that it gets tedious and nobody wants to follow. Some other teams will have the flexibility to stay just with pretty much the set of general rules for governance and so on.


What I found for data governance, even today, even with agents, is still not important for most of the people. For me, it's like key. People is like, "Oh yes, governance, whatever." Especially now with AI, I think we get these problems exposed in a massive way because the agents are exposing these massively in these workflows that has been redesigned in how they are being used, and then you have problems.


For example, one of the things that you could do as a practical thing is, like, okay, when you are redesigning a new workflow on AI, think on your agent. If your agent is Maria, so what kind of access Maria will have? You put there what kind of access, what kind of data will be, what is your role, what is responsibilities? Just create roles and responsibilities for your agent.


Create the permissions based on that. That means that you have there the security that your agent is not going to crawl around your entire database, maybe expose something you don't want to be exposed. Because this has happened quite a bit. In many organizations, that data that shouldn't be exposed is exposed by the agents because they have access to everything. For example, the agent dropping a full database because they have admin rights.


Think of your agent, a part of your workforce, design for them in the AI governance, the responsibilities, the roles, and the access that they will need to have when you're redesigning, this process, and this will also make sure that you are controlled there in terms of governance. Also, on top of that for your people, for your humans in there, automate as much as you can so it's seamless for them, so you don't have to deal with all the old-school type of data governance.


When you have your stewards and these and that, and everybody needs to talk, and you write a policy of 300 pages that no one is going to read, but you already have this embedded on your system. Of course you will still need to write some policies, and you need to tell your people how to write, for example, the standards of your glossary, of your data dictionary, for example. This is much less, and it's less of a burden for them than going through the entire governance process manually.


I think that's the key to be able to do better adoption in data governance. I can tell you that data governance is a slow process. It's going to take more than a couple of years to have a whole program running successfully. However, I think with the agents, and if we do this the right way since the beginning, they are obliged to run a little bit faster in terms of governance. I have hopes on that. Let's see how it goes. I think based on the failures, because literally with agents we are just learning from failure every day, we might be able to tackle this part of governance faster than we used to do with our analytical regular data that we were using day to day.


[00:43:04] Nic: We've walked through this process. We've got confidence in our data. We've got data products that are owned by the people that use them. We've got our governance; we've got our cycle of intelligence. Where do organizations then start when it comes to deciding what to do with that? If we're looking at moving towards investing in agentic use cases, where should they start?


[00:43:37] Jade: That's a good question. It comes back to those value hypotheses and the cycle of intelligence. It's just going back to the fundamentals, like what is it you're trying to influence? Is this a play at growing revenue? Is it a play at reducing cost? How is it going to help your business? What's the outcome? Then once you've got that, I know we're talking about agents and AI, but don't have the kind of AI and agentic tunnel vision; just look through it and think through: what's actually the simplest way to achieve that outcome? What's the fastest way to achieve that outcome?


Just go through that ideation before jumping straight to the agent. Because what you might find in some cases is that it does make sense to use an agent, but in some cases, actually maybe a traditional ML model to do classification is actually the better and more reliable option. I think one thing I always advise is start with the outcome, but then don't only go down the agentic route and just look at the different flavors of AI, the different flavors of automation that could help meet that outcome so that you don't get stuck going straight to an agent.


The other thing is in process, I often see everyone skipping the experimentation phase. When you're building an agentic system, it's much closer to an ML data science workflow. Before you go straight into productionizing or trying to build the system, like many organizations do, just take that step back, frame your different hypotheses, both in terms of tech, in terms of data, and run a time-boxed experimentation phase where you're just validating if this was an agent: "Okay, with the data I have, is it working as I'm expecting?"


You're not building the whole solution; you're just choosing your model, trying with some of the data, just stress testing it a bit, and that'll help you learn and improve before you go into investing in production. What I see happen with clients is they skip that phase, and they go straight into, "Oh, it's really fast and easy to build a thing with an agent in it. Let's just build the agent." Then they find out three, four months down the line when all of that investment has already gone into it when someone spent three, four months working on it, that, "Oh, the agent can't understand which type of asset we're working on, and it gets confused. Now I can't shift it to production because it's not reliable enough." Yes, hopefully I've answered your question.


[00:46:23] Nic: Yes, you have exactly. The experimentation phase is exactly a big part of what I was hoping to hear there. Amy, we're both keen fans of data mesh and the concepts around it. Mentioned very heavily throughout that initial book, and throughout the ecosystem of data mesh is the lean value tree, how that can help organizations prioritize their bets. Can you share more on where you've seen that work and what are some of the basic approaches that people can start taking now to help them better prioritize and prudently invest in agentic initiatives?


[00:47:07] Amy: Yes. Every time that I go to a client to do a workshop, we usually team up also with product innovation teams. We go product, we go AI data strategies, and some AI engineers, for example. We have this program, AWS Catalyst, where we have these in three days; we create a prototype for them, but we go through their use cases.


This is a nice program, for example, because we go on day one, we sit with the teams, the C-levels, with product, with development in general, and we strip down the use cases that they have in mind so whatever they think is the priority.We start stripping them down. We understand what is the value we are bringing from it, what is going to be the return of investment over time, how much tokens is going to cost you along the way approximately, and how you can develop these in a sustainable way that it actually goes into production in the future.


As Jade said, this is an experimentation. We start with this phase, we first prioritize the use cases, we understand what is the one that is going to bring the business value, and then we bring the team, for example, a couple of days, just coding, by coding, understanding a tiny slice of that agent that actually prove that it works and is going to bring value because sometimes people have these picture that whatever agent they think is going to work is the one that is going to solve all the problems.


You need to understand: is this really solving the problem, or is it going to be more costly over time? Doing these small prototypes will help you to understand, okay, it's worth it to continue with the next slices until you create a viable product that you can put into production. This is one of the ways that we do when we have more time in discoveries. We take a little bit longer, so we go with different teams; we run interviews with the different teams.


We try to redesign the whole process as well with product people. We have experienced designers; we design the personas, we try to put the human in the loop, which is very, very important, as Jade mentioned, to give this sense of still ownership to a certain extent and importance to avoid resistance, of course, of these agents over time. We start again with a small, thinner slice of the whole use case. We produce the first iteration, we get feedback again, and we continue building iteratively until we have the production-ready system.


Most of the companies are not really happy of doing the whole process, I will say, because they think it takes too long to bring something into production, but the companies that stick to it, I can tell you that they have the highest chances to have something that is sustainable with a really good return of investment and also waste less tokens and less money over time than the clients that have 100 POCs on the cemetery that nothing came to life. You already spent a lot of money there, but whatever you want to put into production is barely viable, or it will be maybe for a couple of months, as I said at the beginning, with no usage or with a lot of tokens that you cannot even pay because the return of investment is not even good enough to spend that amount of money on that part.


[00:50:29] Nic: Is actually a good point there around the readiness of people's teams and the investment that's going into people as much as is going into technology. I think in our recent CIO survey, as much as CIOs are talking a big game about how they're redesigning workforce workflows, only 24% are really investing in upskilling their staff, and only 15% are investing in their wider workforce. Jade, if a C-suite leader could execute just one action this quarter, what would it be?


[00:51:09] Jade: I think I'd start with adopting the approach with model your value hypothesis. Let's try to lay out that cycle of intelligence and just go through that step by step. What's the outcome you're trying to influence? Get that written down. What do you think will happen by implementing the agent? Maybe just think through: could you achieve that with a simpler implementation so that you are also checking that you're not ending up going straight down the agentic path without looking at other options, but also that you're laying out that value stream so that you have that initiative tied back to the ROI.


That's what I'd say is the action. I will maybe add one more thought just as a closing thought from me, is when we talk about semantic layers and ontologies and the BI semantic layers and taxonomies and all of that, like Amy has mentioned before, none of this is new. None of these are new concepts. From a technical standpoint, this is not something we need to crack. We know how to do it. We've cracked it.


The thing to look out for is don't let it be your next big tech initiative where, because you're struggling with AI, you now go down a big, "Let me create the org-wide semantic layer that has an ontology and has metrics and has taxonomy," because you might not need all of that. Then also, critically, you're not going to address the crux of the issues that you need to address in terms of the ownership, the operating model, the stewardship, because any semantic layer, knowledge layer you put in place is only going to be good if you're continuously curating it and updating it.


You don't want that to be the next big tech initiative that doesn't have ownership tied to it and isn't being curated. Look at solving for data ownership and stewardship as part of bringing that in. First things first, always start to the outcome. The action is map out your value hypotheses, how you're addressing them. Is there a simpler way? Then, more broadly, don't fall into the trap of a big knowledge-based initiative. Stick to what Amy's saying with the lean value tree; go step by step.


It sounds like it's slower, but it gets every organization that does it so much farther and so much more ROI. [chuckles] Yes, that's it for me.

[00:53:51] Nic: Yes, maybe to extend the car analogy, and this might also be stolen from your book, Amy, brakes on a car not to make it go slower, but to make it go faster. Same question to you, Amy. Closing thoughts, and if a C-suite leader could be executing just one action this quarter, what do you think it should be?


[00:54:09] Amy: Don't go big bang; start small, start with one use case that you already did your lean value tree, you know which one is the one that is going to bring value, and start from there and develop all the foundations in this one use case, meaning governance, the ontology, the knowledge graph, whatever you would like to put in there for your agents, and bring this into production.


Once you have these, you have the foundations actually to start on the next use case, and you will know what to do. Again, what Jade exactly said: this is not about doing everything at once because that's going to take years, it's going to be overwhelming, it's going to cost a lot of money, and you might not even use it, but just to start with one thing.


Big bang usually never works. A lot of the errors that happen with-- the problems that happen with the data mesh is that everybody just boom, let's have 10 domains already at the same time. That's why people got stalled, and it never went through. A lot of people say, "Yes, data mesh didn't work." If they had worked with these, with one domain first, and then move towards scaling, data mesh in a lot of organizations will be a reality. Some of them have them, and they are completely working. A lot of them are coming back now on these engagements that I see, like, "Let's fix whatever didn't work three or four years ago." Again, my topic is start small. Start with one use case that is going to bring value. After that, everything is going to be easier. Don't waste your money.


[00:55:41] Nic: Thank you so much. Thank you, both Jade and Amy. I think there's a couple of key takeaways out of today. Just building on that point, I think one of the greatest gaps we see, or the greatest mistakes we see, is thinking that autonomous agents will solve our data debt instead of just amplifying it, which is actually what does happen. Our AI strategies are only as strong as the foundations supporting them. We touched on it there, but we can't forget the people side of this either. Our global CIO survey showed workforce design is now firmly on the agenda.


We need to be ensuring that not only is our data ready for our agents, but our teams and our operating model also need to be ready for our agents too. To our listeners, one challenge for you this week: before you jump into your next meeting, identify one key automated decision or dashboard your team relies on, and ask if an AI agent executed that decision autonomously today, would you trust the data behind it? For more insights on building an agentic-ready enterprise, read the full edition number 39 of Perspectives, "A question of trust: Developing an agentic-ready data strategy." You'll find that at thoughtworks.com/perspectives. Thanks again, Amy.

[00:57:00] Amy: Thank you very much.

[00:57:02] Nic: Thanks again, Jade.

[00:57:03] Jade: No worries.

[00:57:05] Nic: We'll speak to you again next time.

[music]

[00:57:21] [END OF AUDIO]