Key outcomes
By building an AI-powered cybersecurity data platform, the organization:
Reduced 50+ TB of daily security data to fewer than five actionable anomalies.
Simplified onboarding of new data sources through automated workflows.
Accelerated threat investigations with faster access to security insights.
Strengthened the resilience of critical operations and systems supporting UK food security.
Preparing for a new era of cyber threats in retail
As one of the UK's largest grocery supermarket chains, this leading global retailer serves millions of customers every day. Beyond its supermarket operations, the organization also provides banking and mobile connectivity services, making it responsible for a vast and highly interconnected technology estate.
In 2025, coordinated cyberattacks against major UK retailers caused hundreds of millions in losses and disrupted critical supply chains. As threats grew in both scale and sophistication, the organization recognized the need to strengthen its ability to detect and respond to attacks before they could impact operations, customers or food security.
To meet this challenge, the organization partnered with Thoughtworks to build a next-generation cybersecurity data platform.
The challenge: Detecting the signals that matter
Protecting the organization's critical IT assets required detecting threats across an enormous and highly complex technology estate. Every day, its systems generated between 60 and 80 terabytes of security data, making it difficult to identify the signals that mattered most.
The challenge was compounded by the lack of historical threat data. Without previous examples of malicious behavior, the system had no clear reference point for identifying potential attacks. At the same time, data quality issues across source systems made it difficult to reliably analyze information at scale.
To address these challenges, Thoughtworks and the organization set out to build a unified cybersecurity data platform capable of transforming vast amounts of security data into actionable intelligence for Security Operations Center (SOC) analysts.
Our approach: Building a modern cybersecurity platform
Faced with a massive data challenge and tight timelines, Thoughtworks knew that solving the problem would require more than technical expertise. Both the platform and the delivery approach needed to be designed for speed, scale and adaptability.
1. Detecting threats at scale
To successfully uncover threats buried within massive datasets, the solution brought together capabilities across cybersecurity, data engineering and generative AI.
At the core of the platform, Thoughtworks built a custom data ingestion framework and data quality library capable of gathering and cleansing up to 80 terabytes of data per day, preparing it for analysis at scale.
With clean, structured data in place, the team then developed an advanced user and entity-based anomaly detection capability, one of the first of its kind in this space. Instead of looking for known threat patterns, the platform learned to identify unusual behavior that could indicate a potential attack. To ensure reliable outcomes, GenAI agents were continuously evaluated against ground-truth data using an "LLM-as-a-Judge" approach.
The platform was also designed to help security teams act on these insights faster. Automated playbooks were created to enable non-data engineers to easily onboard new data sources, while AI-generated behavioral summaries helped analysts cut through the noise and focus on the threats that mattered most.
2. Democratizing cybersecurity intelligence with Databricks
Databricks on Azure provided the foundation for the platform, enabling the processing of up to 80 terabytes of security data per day while supporting machine learning and GenAI capabilities at scale.
To make the resulting intelligence accessible across the Security Operations Center, Thoughtworks combined GenAI with Databricks Genie to create a natural language interface for analysts. Rather than manually querying complex datasets, teams could ask questions in plain English, investigate suspicious activity and receive immediate explanations.
This reduced barriers to accessing cybersecurity insights, helping analysts make faster, more informed decisions.
3. Delivering at speed
Delivering a platform of this scale within tight, fixed-bid timelines required more than technical innovation. Thoughtworks restructured delivery around parallel workstreams, enabling multiple teams to tackle complex challenges simultaneously and maintain velocity under pressure.
This approach proved critical as the team navigated evolving requirements, scope changes and data quality issues throughout the project. Supported by strong collaboration, shared ownership and engineering practices such as pair programming, the team maintained momentum and delivered the MVP directly into production.
The opportunity went beyond improving threat detection. It was about creating a new capability to help the organization understand and respond to cyber risk. It took a true strategic partnership, combining our engineering expertise with GenAI innovation, to make that happen.
Outcomes: Transforming security data into action
50+ TB of daily data turned into actionable intelligence.
Reducing noise to fewer than five actionable anomalies per day.
Democratized access to security data.
Enabling non-data engineers to onboard and manage new data sources.
Accelerated threat investigations.
Providing analysts with faster access to critical insights
Increased operational resilience.
Safeguarding core operations and the technology systems that underpin UK food security.
Staying ahead of emerging threats
With the platform now in production, the client has established a scalable foundation for its cybersecurity strategy. The platform not only strengthens threat detection today, but also creates the flexibility needed to respond to future threats and evolving business requirements.
Designed as an adaptive GenAI and machine learning ecosystem, the platform will continue to improve as new data sources, signals and behavioral patterns are incorporated, allowing the client to expand its security capabilities over time while maintaining the speed and accessibility that empower Security Operations Center analysts.
