The data mesh solution

As we learned in the Discovering the challenges of modern-day enterprises section, having a central data lake or data warehouse has several disadvantages, especially for large organizations. In the previous section, we learned about changes in data processing requirements driven by ML and advanced analytics and how data now needs to be treated like a product with its own complete life cycle.

To explain a data mesh in one sentence, a data mesh is a centrally managed network of decentralized data products. The data mesh breaks the central data lake into decentralized islands of data that are owned by the teams that generate the data. The data mesh architecture proposes that data be treated like a product, with each team producing its own data/output using its own choice of tools arranged in an architecture that works for them. This team completely owns the data/output they produce and exposes it for others to consume in a way they deem fit for their data. Here...