Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Data Engineering with Apache Spark, Delta Lake, and Lakehouse

You're reading from  Data Engineering with Apache Spark, Delta Lake, and Lakehouse

Product type Book
Published in Oct 2021
Publisher Packt
ISBN-13 9781801077743
Pages 480 pages
Edition 1st Edition
Languages
Author (1):
Manoj Kukreja Manoj Kukreja
Profile icon Manoj Kukreja

Table of Contents (17) Chapters

Preface Section 1: Modern Data Engineering and Tools
Chapter 1: The Story of Data Engineering and Analytics Chapter 2: Discovering Storage and Compute Data Lakes Chapter 3: Data Engineering on Microsoft Azure Section 2: Data Pipelines and Stages of Data Engineering
Chapter 4: Understanding Data Pipelines Chapter 5: Data Collection Stage – The Bronze Layer Chapter 6: Understanding Delta Lake Chapter 7: Data Curation Stage – The Silver Layer Chapter 8: Data Aggregation Stage – The Gold Layer Section 3: Data Engineering Challenges and Effective Deployment Strategies
Chapter 9: Deploying and Monitoring Pipelines in Production Chapter 10: Solving Data Engineering Challenges Chapter 11: Infrastructure Provisioning Chapter 12: Continuous Integration and Deployment (CI/CD) of Data Pipelines Other Books You May Enjoy

Summary

In this chapter, we learned about a data lake and its various characteristics, such as multiple zones, the ability to deal with multiple formats, governance, and—most importantly—its ability to decouple storage and compute.

We also learned about four key data lake architectures: traditional, Lambda, Kappa, and Lakehouse. We analyzed each one from their applicability to certain case scenarios, costing, and overall management.

In the next chapter, I will highlight various tools and services available in Microsoft Azure required to build a data lake using the Lakehouse architecture.

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime}