You're reading from Engineering Data Mesh in Azure Cloud

Product type Book

Published in Mar 2024

Publisher Packt

ISBN-13 9781805120780

Pages 314 pages

Edition 1st Edition

Languages

Concepts

Author (1):

Aniruddha Deswandikar

Preface

1. Part 1: Rolling Out the Data Mesh in the Azure Cloud

2. Chapter 1: Introducing Data Meshes

3. Chapter 2: Building a Data Mesh Strategy

4. Chapter 3: Deploying a Data Mesh Using the Azure Cloud-Scale Analytics Framework

5. Chapter 4: Building a Data Mesh Governance Framework Using Microsoft Azure Services

6. Chapter 5: Security Architecture for Data Meshes

7. Chapter 6: Automating Deployment through Azure Resource Manager and Azure DevOps

8. Chapter 7: Building a Self-Service Portal for Common Data Mesh Operations

9. Part 2: Practical Challenges of Implementing a Data Mesh

10. Chapter 8: How to Design, Build, and Manage Data Contracts

11. Chapter 9: Data Quality Management

12. Chapter 10: Master Data Management

13. Chapter 11: Monitoring and Data Observability

14. Chapter 12: Monitoring Data Mesh Costs and Building a Cross-Charging Model

15. Chapter 13: Understanding Data-Sharing Topologies in a Data Mesh

16. Part 3: Popular Data Product Architectures

17. Chapter 14: Advanced Analytics Using Azure Machine Learning, Databricks, and the Lakehouse Architecture

18. Chapter 15: Big Data Analytics Using Azure Synapse Analytics

19. Chapter 16: Event-Driven Analytics Using Azure Event Hubs, Azure Stream Analytics, and Azure Machine Learning

20. Chapter 17: AI Using Azure Cognitive Services and Azure OpenAI

21. Index

22. Other Books You May Enjoy

Data flow

Data from semi-structured and structured sources is read using Azure Synapse pipelines and written into the bronze layer of the data lake.
Data is then moved between the bronze, silver, and gold layers using more pipelines.
Data from the Medallion storage system is used by Synapse Spark clusters or Synapse SQL pools to further conduct analytics on it.
Analytical data from Synapse is pushed to Cosmos DB and Azure Data Share and read by Power BI to expose the data to various consumers, such as applications, dashboards, and other teams.
The data in Cosmos DB can be searched using Azure AI Search through the mobile app or website.
Power BI surfaces the analytics in the form of dashboards.
Azure Data Share shares the data with external parties that need the data for their processing purposes.