Enable javascript in your browser for better experience. Need to know to enable it? Go here.
Blogs Banner

Taking AI agents from pilot to production starts with workforce design

Do you know how many AI agents are running in your organization right now?


Italo Vietro, VP of Engineering at Parloa, uses this question to gauge whether a company can see what its agents are doing. If you can’t answer it, he says, there’s room to improve your observability. But behind it sits a bigger set of questions: who monitors the agents, who decides when to stop one - and who answers when one gets it wrong.
 

These are workforce design questions, which many CIOs now have to answer. In our Global CIO Survey 2026, 89% of 3,200 CIOs agree they are more responsible for redesigning workforce workflows and labor models than for managing core IT infrastructure. Here, we connect that finding to a Pragmatism in Practice podcast conversation between Italo and Josh Burks, our Global Managed Services Head, on what it takes to move AI agents from pilot to production.


CIOs now own workforce redesign


While most CIOs are taking on greater responsibility for redesigning workforce workflows, our survey also found that 35% feel personally accountable for AI-driven disruption - despite lacking the authority to fully manage it.


Preparation for that responsibility looks thin. When we asked what CIOs are doing or plan to do as technology leadership changes, 24% named upskilling technology staff and 15% named upskilling all staff. A larger share, 51%, plan to embed a technology lead in each business unit.


Rachel Laycock, Thoughtworks CTO, says upskilling is only part of the answer. As AI changes how work gets done, organizations also need to redesign roles, workflows and decision rights. In her words: “Workforce readiness is really about designing the whole system, not just teaching people how to use AI.”


What redesigned work looks like


The podcast shows what that system looks like inside an engineering organization. Josh describes AI shifting technical teams from task execution into an “architectural orchestration mode.” He says engineering and operations teams have spent about 75% of their time clearing tickets, sifting through alerts and writing boilerplate. As agents take over routine triage and code generation, engineers manage autonomous agents, validate complex system behavior and focus on continuous modernization.


Italo sees the same shift at Parloa. His teams think more about the problem space and how to architect agents together to get the outcomes they want. He calls these engineers “product-minded architects.”


Both descriptions address the first question leaders must ask when AI reshapes a workflow: what will people need to know and be able to do?


The operating model and the economics change too


Josh describes managed services moving from labor arbitrage to technology arbitrage, and now to what he calls value arbitrage. Success stops being measured in ticket SLAs and starts being measured in developer innovation capacity, continuous modernization, lower cost to serve and guaranteed business outcomes.


Cost is part of that shift. Our report explains that AI consumption grows as agents call tools, retry tasks and pass work to other agents, so total costs can rise even as individual tokens get cheaper. It recommends asking what it costs to complete a workflow and whether the business outcome justifies it. Italo makes the same point from the engineering side: the wrong model leads to wrong outcomes and high costs, so Parloa routes each task to the model that performs best for it. Josh adds that a control plane has to manage token budgets and access controls.


In the report’s working model, technology and AI leaders make consumption and performance visible, finance connects it to budgets and investment economics, and business owners stay accountable for the value created.


Production needs visibility and clear ownership


Italo’s simple question on the number of agents currently running in an organization demonstrates whether an organization has sufficient observability. Parloa also runs millions of evaluations and simulations on its agents to confirm they do their jobs with enough confidence to deploy, and it runs them in sandboxes with guardrails so a person can pull the plug on an agent that does something it shouldn’t.


Josh recommends a centralized control plane, which we deliver through Agent/works™, built on three foundations: observability and telemetry, governed and reusable skills, and governance covering safety, cybersecurity, token budgets and access controls. Chief Information Security Officers and compliance teams need full auditability over every agentic action, he says. That need explains why 88% of CIOs in our survey say AI adoption is moving faster than governance structures can adapt.


Foundations matter as much as controls. Josh warns that plugging generative AI into fragmented legacy architectures and chaotic data means “you’re just producing bad outcomes faster.” Italo’s advice is clear: “Work on your fundamentals.”


Start with one workflow


Our report proposes joint responsibility for workforce and work design across people, business and technology leaders. Bring those leaders together around one workflow where agents are heading to production, then work through five actions:
 

  • Name the owners: who can change the workflow, who supports affected employees and who is accountable for the outcome.
     

  • Define the new roles: what people will decide, validate and steer once agents take on tasks.
     

  • Count your agents: set up the observability to answer how many are running and what they are doing.
     

  • Test before you scale: run evaluations and simulations on each agent.
     

  • Track cost per workflow: ask finance to connect consumption to the business outcome.
     

For more insights, listen to the full episode and read the Thoughtworks Global CIO Survey 2026: Who governs enterprise AI?

 

Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.

Managed services: Fuel innovation with efficiency