What this book covers
Chapter 1, Balancing the Benefits of Data Lakes over Data Warehouses, explores the evolution of data lakes in the analytical world, and also helps us understand the value of data warehouses.
Chapter 2, Connecting Requirements and Technology, focuses on the architecture of the modern data warehouse and introduces various Azure services, and guides you in choosing the right ones for your needs.
Chapter 3, Understanding the Data Lake Storage Layer, examines the setup and organization of the Data Lake Gen2 storage. You'll learn how to access data and monitor your storage account. You will also learn about backups and disaster recovery, and examine various security and networking options for your storage.
Chapter 4, Understanding Synapse SQL Pools and SQL Options, explores MPP in a cloud PaaS service. You'll also explore the replication and distribution of data in a database. You'll learn about various evolutionary steps of SQL pools in Synapse and other components. You'll also check out various alternative SQL database services in Azure and how you can use them.
Chapter 5, Integrating Data into Your Modern Data Warehouse, shows how to implement ETL/ELT pipelines with Synapse pipelines, or Azure Factory. You'll examine various source connectors and work on integration jobs. You'll also learn how to monitor your integration environment.
Chapter 6, Using Synapse Spark Pools, discusses Synapse Spark pools and how to implement them on Azure. You will examine how to implement notebooks and Spark jobs and integrate additional libraries with your clusters. Finally, we will examine security features and see how to monitor our environment.
Chapter 7, Using Databricks Spark Clusters, examines Azure Databricks. We will learn how to work with it and perform various operations. We'll also learn how to create and use dashboards and run ETL jobs. Finally, you'll learn how to set up Databricks with VNets and implement access control within the workspace.
Chapter 8, Streaming Data into Your MDWH, explores Azure Stream Analytics and how it can be used for analysis. You'll learn how to set up and use the service, and you'll learn about various SQL queries with windowing functions and pattern recognition to detect and highlight various events. You'll also build an online dashboard with Power BI that monitors data streaming in real time.
Chapter 9, Integrating Azure Cognitive Services and Machine Learning, examines various machine learning models that you can use as services in Azure. You'll then explore the Azure Machine Learning service and learn how to implement your own model using the graphical user interface there.
Chapter 10, Loading the Presentation Layer, shows you how to load data into your presentation layer using various tools, such as PolyBase, the COPY command, and Synapse pipelines. We'll also check out how to implement SQL in your data lake. Lastly, we'll explore some options for exchanging metadata between various compute components to improve efficiency.
Chapter 11, Developing and Maintaining the Presentation Layer, examines how to use Azure Synapse, and particularly Synapse Studio, when you implement your presentation layer. You will see how to integrate Azure Synapse with Azure DevOps and how you can automate your deployments. In your role as an modern data warehouse developer, you will also enjoy the developer productivity features that Synapse Studio offers. You'll also dive into disaster recovery and some security aspects of your environment.
Chapter 12, Distributing Data, shows you ways to create data marts to distribute insights in your modern data warehouse with Power BI. You will see how to use Power BI data models and the options to visualize and publish their content and even use the data with other tools. We will also examine Azure Data Share as another option to provide datasets to others.
Chapter 13, Introducing Industry Data Models, showcases various industry data models that you can utilize in your projects using Microsoft's CDM tool. We'll also explore a service in Azure called Industry Data Workbench.
Chapter 14, Establishing Data Governance, takes you through the options that the Azure Purview preview offers for scanning your data and qualifying it. You will see how you can benefit from predefined and custom search patterns and how Purview helps you to find information in your data estate. You will also see how to integrate with other Azure services such as Azure Synapse Analytics or Data Factory.
