A legacy bias towards centralization in the form of data lakes or warehouses is preventing organizations from developing effective approaches to data they need at scale, and from disseminating strategically important information to the teams who need it the most. Consolidation also leads to the creation of architectures and organizational structures that are resistant to change, hampering the enterprise’s ability to test and react to market developments.
To unlock the value of data, enterprises need to break out of the monolith, and make data available to the teams who actually know and use it. Based on the principles of domain-oriented data distribution and treating data as a product that delights its consumers, the mesh is a new approach to data architecture, purpose-built to support a resilient, fast-acting, digital business.
The world of data storage and computation has changed, and companies are no longer necessarily subject to old constraints. Adopting a more distributed data architecture may not be as risky, expensive or time-consuming a proposition as business leaders think. Leading enterprises have built on existing cloud technologies to develop sophisticated platforms that contribute directly to their strategic goals.
Because teams have typically been structured around legacy data resources, moving to Data Mesh inevitably requires organizational change, as well as careful thinking about the connections between data and business strategy. Business leaders may encounter resistance as they redraw the rules on how data is controlled and shared, and how the performance of some teams is measured - but the end is almost certain to justify the means.
In any initiative that touches on data, research shows security and risk management will almost certainly be among executives’ chief concerns. Contrary to perceptions, taking responsibility for data out of the hands of a single, centralized control tower and ‘federating’ ownership among domain teams creates positive security and governance impacts, by assigning control of the data to those who know it best, and fostering a broader focus on data accuracy and privacy.
While fretting about data skills remaining in short supply, many companies have yet to recognize the resources already at their disposal. Platforms and models are evolving in a way that’s making them more user-friendly, and opening the possibility of training non-specialists in the basics of fields like data engineering and data science. Data Mesh supports this trend by emphasising accessibility and consumer experience - and, sometimes, allowing people to break things.
The acceleration of data demands will continue to present challenges for businesses, but developments in architecture are just one reason for optimism about what comes next. Computing and hardware are evolving at an incredible rate, and there are pools of insight-rich data that remain largely untapped. In this race, fearless thinking and a willingness to strike partnerships will help businesses claim the rewards.