Goals of a modern data platform
In Chapter 15, Implementing a Data Mesh Strategy, we discussed how a central data platform team is responsible for building a platform that makes it easy for both data producers and data consumers to work with organizational data.
A data platform is intended to provide a system where multiple teams from across an organization can easily ingest data (including both structured and semi-structured, via batch and streaming), process the ingested data, and create new data products by joining datasets. It should also provide data governance controls, a catalog for making data discoverable across the organization, and the ability to easily share datasets across different teams/data domains.
Let’s review some of the top goals for a modern data platform, after which we will explore approaches to building these data platforms.
A flexible and agile platform
As we all know, the only constant is change. We have seen this throughout this...
