The predictability gap in digital public infrastructure

Key takeaways:

1. Code is easy. Alignment is hard.
2. Adoption is a complex behavior, not an engineering task.
3. Measure trust, not just transaction volumes.
4. GenAI is an accelerator for implementation.
5. Simulate before you launch.

In some countries,  digital public infrastructure (DPI) has moved from pilot to national dependency. In others, it remains politically contested, unevenly adopted, or stalled entirely. Much of  the global conversation has focused on engineering design — often overestimating how far technical brilliance alone can carry adoption. Elegant protocols for identity, payments, and data exchange have created the rails on which a more inclusive digital economy can run. In many implementations, a gap persists — but its causes and severity vary significantly by country..

That gap has a name: the predictability gap. And until recently, it felt like a permanent condition — something to be managed, never closed.

Something has shifted. Generative AI introduces new capabilities that may change how some DPI ecosystems are built and managed — but its impact will depend on institutional readiness. The promise is undoubtedly alluring: the ability to compress a 10-year implementation slog into a two-to-three-year sprint. Not by shortcutting the hard problems. By finally giving us the tools to see them clearly, simulate them honestly, and act on them at the speed that complex human systems actually demand.

Here, we explore  that possibility. But to understand it, we first need to be honest about the gaps — the structural, cultural and institutional challenges that elegant code alone has never been able to solve.

The hard truth about DPI

The paradox at the heart of every protocol

DPI rests on a deceptively simple architectural insight — what engineers call the Hourglass Model. Standardize a thin layer of core protocols for identity, payments and data exchange, and you liberate everything above and below. Fintechs can innovate without permission. Governments can interoperate without rebuilding from scratch. Citizens can participate without friction.

The model is technically sound and has proven viable in controlled environments.

The paradox is that the thinner the technical waist, the thicker the institutional burden becomes. A centralized platform controls the full stack: it can enforce compliance, set quality standards, and course-correct in real time. A protocol-based DPI hands those responsibilities to a self-organizing ecosystem of banks, fintechs, government agencies, and citizens — none of whom share the same incentives, timelines or appetite for risk. The engineering problem often becomes more tractable, while coordination complexity increases — sometimes dramatically.

Takeaway 1: Code is easy. Alignment is hard.

The shift from platforms to protocols has de-bottlenecked the engineering layer. But it has transferred the complexity upward — into institutions, into culture and into the messy, non-linear behavior of millions of human beings. That is where the predictability gap actually lives.

Leaders who recognize this early build differently. Instead of asking "Is the protocol ready?" they ask "Are the institutions, incentives, and trust mechanisms ready to carry this protocol forward when the technical team moves on?"

Why adoption isn’t an engineering problem

The category error that kills most DPI programs

The intuitive assumption — make something simple, and people will use it — is only half true. India’s Unified Payments Interface (UPI) three-tap payment flow is a masterclass in interaction design. Brazil's Pix is instant, elegant and free. Estonia's X-Road is a model of interoperability. Yet simplicity of interface is not the same as simplicity of adoption.

Dave Snowden's Cynefin framework offers the most honest diagnosis available. Building the technical architecture of a national payments system is a complicated problem: one that requires deep expertise and skilled engineers who follow a structured plan. Getting a fish vendor in rural Tamil Nadu to trust a QR code more than the cash she has handled her whole life — that is a complex problem. Cause and effect are only visible in hindsight. The intervention that worked beautifully in Mumbai may actively backfire in Varanasi.

When dealing with complicated problems, you must first sense, analyse, then respond. In a complex one, you must probe first — run safe-to-fail experiments, watch what emerges, then respond and amplify what works. Rigid 12-month plans are the wrong tool for a complex challenge. What is needed is a culture of rapid experimentation, honest feedback and the institutional courage to change course quickly.

Takeaway 2: Adoption is a complex behavior, not an engineering task.

Getting millions of citizens to trust a new system is non-linear. There is no algorithm for it. What there is, is a discipline: probe with small pilots, sense what is actually happening on the ground (not what the dashboard says), and respond fast. Probe-sense-respond isn’t a methodology, it’s a mindset shift — and for many government institutions, it’s a profoundly uncomfortable one.

The Pendulum Trap: Why copy-paste fails

When governments observe a successful DPI — India's UPI, Estonia's X-Road, Brazil's Pix — they naturally want to replicate it. They study the architecture. They commission the same vendors. They announce the same timelines. And then, more often than not, they are puzzled when the results don’t follow.

This is the Pendulum Trap. Estonia did not build X-Road because it had a brilliant technology strategy. It built X-Road because the Soviet collapse left it with no legacy infrastructure to protect — and an unusually reform-minded government made a conscious, existential bet on digital-first governance. India's Aadhaar did not succeed because of its biometric design; it succeeded because Nandan Nilekani had the political access to make enrolment a condition of benefit transfer, and because a moment of political urgency around financial inclusion aligned the incentives of an otherwise fragmented bureaucracy.

Three gaps that persist — and  their cost

Beneath the broad challenge of adoption lie three structural gaps that are common across DPI programs that has struggled. Understanding them is the first step to designing around them.

Gap 1: Coordination without a choreographer

Open protocols are, by design, nobody's problem. That is their genius — and their fatal weakness. India’s Open Network for Digital Commerce (ONDC) is a vivid illustration: the protocol is genuinely transformative, but aligning logistics providers, sellers, buyers, payment gateways and dispute resolution mechanisms — all operating on different timelines and with different incentive structures — creates coordination overhead that rivals the original engineering effort.

Every time you solve one bottleneck in the value chain, the bulge reappears somewhere else, almost always in the governance layer. Nobody owns the outcome in a protocol ecosystem. That diffusion of responsibility is what makes scaling so persistently hard.

Gap 2: The KPI crisis — measuring the wrong things

Most DPI programs are measured on what is easy to count: total registered users, transaction volumes, APIs deployed. These lag indicators are satisfying to report and politically convenient. While they might be useful proxies  for charting progress, they’re insufficient as governance instruments on their own.

A system can show 500 million registered users and still be failing if 60% transacted once and never returned. It can show exponential growth in transactions and still be systematically excluding the 200 million people it was specifically designed to serve.

These indicators are harder to collect and politically less convenient. They ‘re also the only ones that tell you whether the system is actually working.

Gap 3: The sustainability paradox

DPI operates on a five- to 10-year horizon. Governments operate on three- to five-year electoral cycles. The result is what researcher Richard Heeks calls 'successful failure': IT projects delivered on time and budget, where the social outcome never materializes. Nigeria's technology governance body reports that 56% of government IT projects fall short — not because the technology failed, but because the institutional scaffolding was never built.

The systems that survive political transitions are those institutionalized deeply enough that no incoming administration can dismantle them. Brazil's Pix survived because the Central Bank — not a political ministry — owned the mandate. Estonia's X-Road survived multiple government changes because it was embedded in law and in the daily operational dependency of every major state institution.

For most DPI programs, the sustainability question is never asked at design time. It surfaces only in the post-mortem.

Takeaway 3: Measure trust, not just transaction volumes

The shift from lag to lead indicators isn’t a measurement exercise — it’s a governance transformation. When a team is held accountable for merchant trust scores and grievance resolution quality, not just transaction counts, it makes fundamentally different decisions. It invests in community outreach. It fixes the problems that real users actually experience, not the ones that show up on a dashboard.

Culture Is the hardest infrastructure

Why the translation layer determines everything

There is a function in every successful DPI program that the architecture documents almost never specify: the continuous, active translation between the technical design, the institutional constraints of government, and the actual behavior of the citizens the system is meant to serve. When that function works well, adoption compounds. When it is absent, even technically elegant systems stall.

The instinct is to look for a single exceptional individual to carry this function — and point to a Nilekani or a Viik as proof that great DPI requires a heroic champion. But the UK’s National Programme for IT had a single, forceful individual at its helm too, and it became one of the most expensive technology failures in public sector history. The lesson is not “find the right hero.” Exceptional individuals sometimes create the conditions for this translation function to operate — but they are not the function itself, and they are not a scalable or reliable mechanism for delivering it.

What DPI programs actually need is a durable institutional mechanism — a structured role, embedded in governance, that is explicitly accountable for translating across three worlds simultaneously: the world of technical protocol design, the world of bureaucratic and political constraint, and the world of citizen behaviour and community trust. Call it the reformer-translator function. It requires technical literacy, institutional credibility, and deep community knowledge. Rarely does one person hold all three in equal measure. What successful programmes build, instead, is a small, stable team with complementary strengths across each domain — operating with a clear mandate, real decision rights, and direct lines to both the technical architecture and the political leadership. The goal is not to find a hero. It is to make the translation function so structurally robust that it does not depend on one.

Collectivist cultures need evangelism, not just marketing

Technology adoption looks fundamentally different depending on the cultural context. In individualistic societies, people make independent decisions based on clear value propositions — a good UI, a compelling price, an obvious benefit. Reach them well, and adoption follows.

But most DPI deployments are happening in collectivist cultures. And there, adoption works entirely differently. People do not adopt because the technology is objectively better. They adopt because someone they trust — in their neighbourhood, their church, their community group — has already adopted it and vouched for it. The quality of the interface, although important, is not everything. The quality of the social proof is quite important.

 Trust is the invisible substrate of DPI — and it is in short supply. The OECD's 2024 survey of 60,000 citizens across 30 countries found that more people distrust their national government than trust it. The Edelman Trust Barometer, tracking 33,000 respondents across 28 countries, found that trust in government has fallen by more than 10 percentage points since 2019. In the markets where most DPI deployments are currently happening — Kenya, Nigeria, India, Indonesia — institutional trust ranges from fragile to actively contested. Any deployment that does not explicitly design for trust — through transparency mechanisms, accessible grievance redress, and visible accountability — is building on sand

Simulate before you launch

Agent-based modelling and the digital pre-mortem

Complex adaptive systems, by definition, cannot be fully predicted. But they can be probed — and the tools for probing them intelligently have become dramatically more powerful.

Agent-based modelling (ABM) is a revolutionary tool for DPI planners. Rather than asking 'what will happen when we launch this protocol?', ABM asks a better question: 'what behaviors emerge when millions of heterogeneous actors — each with their own trust threshold, social network, and incentive structure — interact with this protocol over time?'

A simulated fish vendor in rural Tamil Nadu following the rule 'I adopt a payment tool when three people I trust have already adopted it' generates very different adoption curves from an urban merchant following 'I adopt when the transaction fee is below 0.5%'. Running thousands of such simulations, with different cultural parameters, trust calibrations, and external shocks, allows policymakers to conduct digital pre-mortems — identifying failure modes before they become irreversible.

Takeaway 5: Simulate before you launch.

Population-scale behavior is prone to external shocks and unintended consequences that no linear plan can anticipate. ABM does not eliminate uncertainty — but it dramatically reduces the number of surprises that turn into catastrophes. A digital pre-mortem is not pessimism. It is the most optimistic thing a DPI leader can do, because it means arriving at launch day having already solved the problems that kill most programs.

GenAI as an execution accelerator for DPI 

From a 10-year slog to a two- to three-year sprint

Here is the creative leap that changes everything: generative AI doesn’t solve the predictability gap. It gives us, for the first time, the tools to navigate it — at the speed and scale the problem actually demands.

Think of DPI primitives as universal verbs: register; identify; pay; consent; book; certify. These six verbs, encoded as open protocols, can compose most workflows a modern economy requires. But verbs without grammar are just vocabulary. Generative AI becomes the grammar — the orchestration layer that composes these primitives into sentences, paragraphs, and ultimately into the living documents of a functioning ecosystem.

Platforms like AI/worksᵀᴹ — Thoughtworks' Agentic Development Platform — are beginning to demonstrate what this looks like in practice. API specifications and protocol documents turned into functional prototypes in days, not quarters. ABHA V3 health account enrolment, Beckn logistics flows, Finternet domestic transfer protocols — each built from source specifications into working backends, frontends, and tests, with every API call traceable to its exact specification section. This is not aspirational. It is happening now.

Three ways GenAI compresses the implementation cycle

1. Compressing the coordination tax

The coordination overhead in a DPI ecosystem is fundamentally an information problem. Participants cannot align because they cannot see the same picture. GenAI can construct a shared operational memory — a living institutional intelligence that tracks ecosystem state in near-real-time, surfaces conflicts before they become crises, and generates coordination artefacts that today require months of committee work. Imagine an ONDC governance model where emerging misalignments are surfaced to human decision-makers within hours, not quarters. The humans still decide. The machine handles the cognitive load of seeing the whole system clearly.

2. Reinventing the KPI stack

GenAI makes it possible to measure what actually matters — not because the data was unavailable before, but because the synthesis required to turn it into actionable insight exceeded human cognitive capacity at the required scale. An AI-powered monitoring layer that flags adoption anomalies in near-real-time — a sudden dropoff in a specific district, a surge in grievances from a particular demographic — changes the governance posture from reactive to predictive. For the first time, lead indicators become operationally feasible at population scale.

3. Compounding returns on implementation investment

In the traditional DPI model, each country deployment is a fresh cost. Architecture decisions made in Peru don’t reduce the delivery cost in Brazil. But when DPI primitives are encoded into a reusable AI-powered platform, each deployment enriches the library. The first client pays full implementation cost. Our hypothesis is that, by the fifth implementation, approximately 50-70% could be pre-built and reusable across regions. The same consent framework that deploys DEPA-based health data sharing in India can, with configuration rather than reconstruction, deploy GDPR-compliant consent for clinical trials across 50 countries. DPI primitives, amplified by GenAI, earn compound interest on every implementation.

Minimum viable governance: The right governance for the right speed

GenAI's pace creates a governance challenge that mirrors the DPI paradox itself. Traditional governance frameworks assume stable technologies and predictable consequences. GenAI transforms faster than conventional mechanisms can adapt — creating a dangerous lag between capability and accountability.

MIT CISR's concept of minimum viable governance (MVG) offers a practical response: the least amount of governance required to manage risk effectively while preserving the capacity to sense and seize opportunities. MVG replaces heavy compliance committees with adaptive guardrails — decentralised decision rights within high-trust boundaries, reviewed and updated on sprint cycles rather than annual policy reviews.

The parallel to DPI governance is exact. Just as DPI requires embedding compliance into the protocol rather than enforcing it through manual oversight, MVG embeds accountability into the operating rhythm rather than delegating it to a governance function that inevitably lags the technology.

The era of 12-member weekly committee meetings that run for hours and consume reams of paper will give way to automated systems operating silently in the background, generating alerts only when necessary. These systems will need only minimal human oversight, freeing regulators to focus on the future and on creating enabling environments for ecosystem improvement and enhanced welfare offerings.

Takeaway 4: GenAI is an accelerator for implementation.

The organizations that will lead the next decade of DPI are not those with the most elegant protocols. They are those that pair protocol clarity with GenAI-powered execution capability — and that have the institutional courage to govern at the speed the technology demands. MVG is not a shortcut. It is the only governance model that can keep pace with the problem.

What leaders must do differently

Four commitments for the next decade of DPI

The predictability gap is not closeable in any final sense — it is a characteristic of complex systems, and it must be managed continuously. But the leaders who navigate it successfully share four commitments that go well beyond technical competence.

1. Treat adoption as the primary engineering challenge

Not the API. Not the identity layer. Not the data exchange protocol. Those are prerequisites. Adoption is the product. This means resourcing evangelism at the same level as engineering, measuring lead indicators (community trust scores, grievance resolution rates, merchant training completions) rather than only lag indicators, and treating cultural integration as a design constraint from day one — not as a communications task for month 18.

2. Build institutions, not just systems

Every DPI program should ask one question at design time: what would make this system survive a change of government? If the answer is 'nothing', the program is fragile by design. The X-Road lesson is not about interoperability architecture — it’s about legislative anchoring, operational dependency, and cross-government buy-in that made the system politically undismissable. Build for the post-mortem you hope never to need.

3. Manage in cadences, not in plans

Monthly reporting cycles are too slow for complex adaptive systems. Quarterly reviews are archaeological. The discipline of weekly accountability cadences — each cycle surfacing what worked, what didn’t, and what the team will do differently next week — isn’t a project management technique. It’s the operational mechanism that allows probe-sense-respond to function. Plans tell you where you intended to go. Cadences tell you where you actually are.

4. Use AI to compress, not to substitute

GenAI accelerates sensing and learning cycles. It doesn’t replace the human judgment required to interpret what signals mean, or the political will required to act on them. The most effective DPI leaders will be those who can hold both in the same hand — the speed of AI-driven sensing and the wisdom of human-driven sense-making. AI is the accelerator. Leadership is still the pilot.

Possibility, responsibility and the road ahead 

The primitives are proven. The protocols are mature. The engineering capability — now amplified by Generative AI — is sufficient to build DPI ecosystems at a pace and cost that was genuinely impossible three years ago.

What remains is the will to close the predictability gap on its own terms: not by pretending it is a technical problem, but by treating adoption, coordination, sustainability, and cultural integration as the primary engineering challenges they have always been.

The code has never been the hard part. The hard part has always been trust. The alignment. The human beings at every point in a system have to choose, day after day, to participate in something larger than their immediate incentive to opt out.

Generative AI does not eliminate that challenge. It gives us, finally, the tools to see it clearly enough to address it — and to compress the years of iteration required to get it right into a timeline that matches the urgency of the problems DPI is designed to solve.

None of this is to say that DPI is unambiguously good, or that its expansion should be treated as self-evidently desirable. The same identity infrastructure that delivers welfare payments to the rural poor can become a surveillance apparatus in the hands of an authoritarian state. The same consent framework that protects health data can be designed — or quietly redesigned — to extract it. The same payment rails that include the previously unbanked can exclude those who cannot or will not participate in a digital system. These are not hypothetical risks. They are documented realities in deployments across multiple continents. The obligation on DPI leaders is therefore not just to build systems that work — it is to build systems that are worthy of the trust they ask citizens to extend. Transparency, grievance redress, independent oversight, and the genuine right to opt out are not features to be added later. They are the conditions under which DPI earns its legitimacy in the first place.