What is Data Mesh?

A domain-driven analytical data architecture where data is treated as a product and owned by teams that most intimately know and consume the data.

Today, data is ubiquitous. Data is the by-product of any and every action we take. Everything, every system, every process, every sensor generates data. Technology makes it easier for organizations to collect and store data, for businesses to leverage to make better decisions or create more tailored experiences for their customers.

However, organizations are struggling to enable and empower their employees to make the most informed and timely decisions possible. Centralized data platform architectures fail to deliver data with the speed and flexibility scaling organizations need. Data Mesh is a response to this problem.

Data Mesh applies the principles of modern software engineering and the learnings from building robust, internet-scale solutions to unlock the true potential of enterprise data.

Domain ownership

Modern digital businesses have evolved to organize into product teams aligned with business domains but still manage analytical data centrally. The Data Mesh approach allows product teams to own, control and have the accountability to share the data they produce — in a controlled and compliant way. This approach brings together data knowledge to speed up the delivery of value with data. How can the components of the data ecosystem and their ownership be decentralized? What capabilities might be needed to enable domain teams to create and manage their data products?

Data as a product

A shift to product thinking for data enables better data quality by creating accountability with the data product owner. Instead of treating data as an asset to accrue, data becomes a product: it is something that has a “customer” and the aim is to delight those customers — the data scientists and data analysts. Data products should be cohesive and independent in the domain context. So What does it mean to be a data product owner? Who can be a data product owner? How do data product owners collaborate?

Self-serve data platform

Access to data typically requires multiple conversations, tickets and approvals. Data Mesh seeks to eliminate the friction to deliver quality data for producers and enable consumers to discover, understand and use the data at rapid speed. This approach ensures compliance and security and can significantly lower the lead time to get insights from data. What does it take to make data sets independently understandable? How do you ensure appropriate access controls over the data? How would you manage the cost of infrastructure on-prem or in the cloud?

Federated computational governance

Traditional data platforms tend to default to centralized data governance. Data Mesh requires a federated computational governance model that maintains global controls while increasing local adaptability. Semantic standards, security policies and compliance are managed centrally while the responsibility to comply is decentralized to data product owners; the policy execution in the context of each and every data product is enabled computationally by the platform. How would data governance units be run? How are data product owners organized to ensure accountability? How are security policies enforced?