Data mesh concepts

If you recall from Chapter 8, Data Sharing, we kept the important topic of a distributed data lake that spans multiple AWS accounts open-ended. Now is a good time to complete that story. Even today, the vast majority of use cases that require a data lake can be solved by building a centralized data lake. However, as organizations become bigger, new lines of businesses (LOBs) that work as autonomous units become a reality. All these LOBs add more data sources to grow their business units, resulting in the exponential growth of data at the enterprise level.

Sharing data within an enterprise presents its fair share of challenges. Different LOBs have invested in cloud-based data lakes, along with customized analytics solutions, tailored to address their specific business needs. However, these systems are often designed to cater to particular types of data and may not seamlessly translate to other problem domains.

For many large organizations with many LOBs, a centralized...