Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Cloud Scale Analytics with Azure Data Services

You're reading from  Cloud Scale Analytics with Azure Data Services

Product type Book
Published in Jul 2021
Publisher Packt
ISBN-13 9781800562936
Pages 520 pages
Edition 1st Edition
Languages
Author (1):
Patrik Borosch Patrik Borosch
Profile icon Patrik Borosch

Table of Contents (20) Chapters

Preface Section 1: Data Warehousing and Considerations Regarding Cloud Computing
Chapter 1: Balancing the Benefits of Data Lakes Over Data Warehouses Chapter 2: Connecting Requirements and Technology Section 2: The Storage Layer
Chapter 3: Understanding the Data Lake Storage Layer Chapter 4: Understanding Synapse SQL Pools and SQL Options Section 3: Cloud-Scale Data Integration and Data Transformation
Chapter 5: Integrating Data into Your Modern Data Warehouse Chapter 6: Using Synapse Spark Pools Chapter 7: Using Databricks Spark Clusters Chapter 8: Streaming Data into Your MDWH Chapter 9: Integrating Azure Cognitive Services and Machine Learning Chapter 10: Loading the Presentation Layer Section 4: Data Presentation, Dashboarding, and Distribution
Chapter 11: Developing and Maintaining the Presentation Layer Chapter 12: Distributing Data Chapter 13: Introducing Industry Data Models Chapter 14: Establishing Data Governance Other Books You May Enjoy

Using data lineage

Once the data factory is connected, it will send lineage information into your Purview environment for every pipeline that is run. Give it a try and create a Data Factory pipeline that copies data from one folder to another in your data lake. Remember: you are quickest when you use the Copy Data Wizard (or just use the MyFirstPipeline pipeline that you created in Chapter 5, Integrating Data in Your Modern Data Warehouse, if you used the data factory there).

When you are finished in the data factory, switch back to Purview, repeat your scan (again, this might take a few minutes), and search for the newly created file or the pipeline name, and in the asset details, check the Lineage tab:

Figure 14.22 – First lineage overview for a Data Factory Copy pipeline

When you check the lineage closely, you will see that you can drill down to the column level and reveal even the column mappings.

Imagine the power of this feature, when you...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime}