Getting to 82% Renewables:
How AI secures our energy future

The Australian National Electricity Market (NEM) is now ground zero for the global energy transition. Driven by the 82% renewables target by 2035, and the rapid proliferation of distributed energy resources (DERs) like rooftop solar and batteries, grid stability is no longer a fixed engineering problem, but a real-time data challenge. Across boardrooms and policy roundtables, “AI-driven energy” has become a recurring theme in addressing the energy trilemma of sustainability, security and affordability. For leaders, however, the key questions are no longer about what AI can do. The focus has shifted to how it can be deployed safely, effectively and in a way they can trust, as well as why it matters for long-term competitiveness, regulatory compliance and grid resilience.

There is a growing gap between the hype surrounding AI and its practical application in the energy sector. The organizations achieving real progress are those that start with business-critical problems, establish a strong foundation in data and governance and then scale AI initiatives systematically.
 

Systemic bottlenecks: The real inhibitors

Despite the necessity of AI, many initiatives remain stalled because enterprises fail to address the core challenges of rapid technological shifts, data gaps and system resilience in the Australian market. The real inhibitors are not the algorithms themselves, but the following systemic challenges:

1. Data fragmentation vs. regulatory urgency
 

The challenge is no longer about finding data; it's about making it complete, reliable, real-time and unified for AI consumption. Energy companies in Australia are operating in a mixed environment of decades-old, siloed data systems and cutting-edge platforms.
 

This fragmentation directly conflicts with increasing regulatory pressure for transparency and interoperability. With frameworks such as the Consumer Data Right (CDR), the AEMC’s smart meter interoperability standards and critical infrastructure reporting mandates, regulators now expect energy data to be accurate, traceable and available in near real-time.
 

The constraint: This fragmented data ecosystem creates a direct bottleneck for scaling AI. Without the ability to easily integrate these diverse sources, firms cannot move from isolated predictive maintenance pilots to complex, real-time grid optimization. This data debt directly impacts the growing transmission base's ability to manage new assets and comply with regulatory mandates efficiently.
 

2. The IT/OT convergence chasm

The shift to a dynamic, two-way grid requires integrating Information Technology (IT) and Operational Technology (OT), an undertaking complicated by a massive influx of regulatory-driven data and the need for new simulation environments. This convergence is non-negotiable for delivering core services like predictive asset maintenance and optimizing network capacity, in particular for generation, transmission and distribution companies who need to manage assets without exponentially increasing cost to the end customer.
 

The constraint: Systems must be built to ingest, govern and act on this data within seconds to successfully feed digital twin ecosystems. This convergence is non-negotiable for enabling Distributed Energy Resource Management Systems (DERMS) and Vehicle-to-Grid (V2G) capabilities, which are essential for achieving net-zero stability and simulated operational security.
 

3. Forecasting shortcomings
 

The trading division of the gentailers is where data science comes to life and determines the organization's ability to predict the demand and supply of energy. Traditionally, with stable energy sources, this was an industrialized process with a reliable track record. However, the introduction of renewables significantly increases the unpredictability and intermittency of energy sources, leading to fluctuations in power generation, grid instability, energy waste and commercial loss.
 

The constraint: Models need to evolve to cater for these more dynamic elements by embedding AI-driven predictive capability, advanced machine learning and deep learning models. This is required to analyze large datasets of meteorological data and historical performance, thereby aligning production with consumption and minimizing commercial losses.

4. The AI talent and trust gap
 

Scaling high-stakes, AI-driven controls requires both scarce expertise and operator confidence. A talent constraint presents a rising challenge, driven by a shortage of engineers with the rare blend of software engineering excellence and deep OT/energy domain expertise. Beyond capability, energy organizations face crucial trust hurdles as operators must overcome a lack of confidence in the AI's "black box" decisions, particularly in mission-critical scenarios.
 

The constraint: Organizations must shift from simply acquiring technology to embedding transparency into the platform itself by adopting a collaborative solution design approach to ensure system security and reliability are maintained at the operational level. It also includes embedding governance and regular review of model accuracy to account for inevitable model ‘drift’ or redundancy as more advanced models become available.
 

5. Grid security and increased cyber vulnerability
 

The vulnerability of the grid is being amplified at the edge. Hundreds of thousands of new DERs are connecting to the network without mandated, unified cybersecurity standards or API integration integrity checks in place. This creates a vast, un-audited attack surface at the edge of the grid.
 

The constraint: This risk is compounded by the IT/OT convergence. The integration of historically air-gapped OT systems (grid controls, SCADA) with IT services exponentially increases the threat of digital compromise, leading to physical blackouts. Deploying AI at the edge for real-time control requires security to be built in, not bolted on. Leaders need a ‘zero trust’ architecture across the digital fabric to manage device identity and control-system integrity, turning security from a compliance checklist into an enabling capability.
 

Actionable roadmap: Practical recommendations for leaders
 

To move beyond AI hype and ensure adoption creates measurable outcomes that address the core challenge of balancing net-zero speed with system resilience, several decisive actions stand out:
 

1. Anchor decisions in business criticality
 

Start all initiatives with the most pressing, high-stakes operational or commercial problem. Anchor AI development not in model complexity, but in specific, quantifiable outcomes - such as asset reliability, workforce optimization, or customer retention - before selecting models or tools.
 

At Thoughtworks, we use value stream mapping and domain-driven design to target the highest-impact business problems surgically. This ensures every (AI) initiative is directly tied to measurable commercial outcomes, preventing 'solutionism' and focusing engineering effort where the net zero challenge is most acute.
 

2. Prioritize adoption and organizational Trust
 

AI delivers value only when it is successfully embedded in daily workflows and trusted by operators. Clear communication, active operator engagement and transparency are critical for building confidence in AI-driven decisions (especially in the control room). Leaders must invest as much in upskilling their workforce and collaborative design as they do in the technology itself.
 

The Thoughtworks approach: Our multidisciplinary teams, which blend software engineers, data scientists and experienced designers, work collaboratively with your domain experts. We use lean-agile methods to co-design AI and engineering solutions, building transparency and explainability into the user interface. This collaborative approach builds the organizational trust necessary to move from monitoring to AI-driven control, fundamentally de-risking operational adoption.
 

3. Establish a unified data foundation
 

Integrating siloed, legacy and modern data into a single, governed architecture is the non-negotiable prerequisite for scaling AI. This shift, often delivered via data mesh or lakehouse patterns, enables the enterprise to treat data as a secure, productized asset, essential for transitioning from simple predictive maintenance to complex, real-time grid optimization.
 

We pioneer a data mesh approach to break down legacy data silos, treating data as secure, discoverable and immediately usable data products. By implementing the data mesh, we de-fragment the data landscape. This approach decentralizes data ownership to operational domains (e.g., grid operations, customer metering). It provides the secure, self-service and high-fidelity real-time data platform required for scaling complex AI models across the enterprise.
 

4. Adopt an evolutionary, platform-first approach
 

The pace of the energy transition demands continuous delivery, not monolithic, multi-year projects. Leaders must embrace platform engineering to create robust, internal digital platforms that serve as secure, self-service infrastructure for AI development. This platform-as-a-product model accelerates the pace of innovation, reduces the risk of vendor lock-in and ensures the utility is structured for sustained, rapid technological mandates.
 

Thoughtworks introduced Evolutionary Architecture thinking and are global leaders in Platform Engineering. Thoughtworks architects and builds internal, self-service developer platforms that abstract away the complexity of cloud, security and infrastructure. This enables engineering teams to deploy new AI models for forecasting, control and asset management in days and weeks, rather than months, delivering a decisive, sustained advantage in the race to Net Zero.

Value now: Where AI delivers ROI
 

In the Australian energy sector, AI has moved past theoretical pilots into four areas where current applications are delivering measurable, high-impact value today:
 

1. Retail customer experience and reducing cost-to-serve
 

Value now: Retailers are using AI to analyze vast customer consumption data, market prices and personal preferences to comply with regulatory requirements, improve customer service and cross-sell unregulated services (such as broadband or mobile plans, streaming and EV car subscriptions, solar and battery bundles or V2G bundles).
 

Business outcome: Reduces cost-to-serve through automated support and personalized tariff (by customer segment) recommendation engines. It also drives customer acquisition and improves retention by facilitating programs like dynamic pricing and participation in solar battery and V2G schemes.

2. Asset optimization and predictive maintenance
 

Value now: Generators, transmission and distribution operators are applying AI models and digital twins to process real-time sensor data from assets. This enables predictive maintenance that identifies potential equipment failures and degradation before they occur. Australian distributors like Ausgrid and Essential Energy are already leveraging digital twins to simulate network capacity, predict asset failure and optimize disaster response.
Business outcome: Reduces unplanned outages, increases asset reliability and lowers operational expenditure (OpEx).
 

3. Field force efficiency and safety
 

Value now: Operational teams are using machine learning to optimize the logistics of large, dispersed field crews.
Business outcome: Improves workforce safety and operational efficiency by automating optimal scheduling, route planning and resource allocation. This is essential for quickly responding to faults across Australia's vast geographic spread.
 

4. Grid system security and forecasting
 

Value now: Grid operators (like AEMO) are using AI-enhanced forecasting to manage the immediate impact of variable renewable energy and design the control room of the future to manage grid stability and security.

Business outcome: Provides enhanced situational awareness and real-time load balancing capabilities, which are crucial for maintaining system frequency and stability as coal generation retires.

The future leap: Scaling value and system resilience
 

The greatest opportunity for competitive advantage lies in scaling these isolated applications into an integrated, AI-driven energy system. Addressing the data fragmentation highlighted earlier unlocks three future capabilities:

Final thought: Mastering the execution imperative
 

AI has the potential to be a defining factor in balancing decarbonization, resilience and affordability in the energy transition. This is not about chasing the most advanced models; success will only come from systematically aligning AI with business priorities, data readiness and human adoption. The next phase of AI in energy is no longer about experimentation - it is about moving from concept to competitive advantage by mastering the required engineering. Thoughtworks is the partner for this execution, bringing world-class platform engineering expertise to de-risk the most complex challenges, from building secure, governed data mesh foundations to enabling IT/OT convergence. We are here to ensure that your commitment to net zero translates into a continuously delivered competitive advantage.

References:

1 AEMO (2025), 2025 Inputs and Assumptions Workbook, pg 109 - 113; AEMC (2024), Data Centres and Energy Demand – What’s Needed?
2 https://epoch.ai/trends
3 https://epoch.ai/data-insights/nvidia-chip-production