You're reading from Data Engineering with Google Cloud Platform - Second Edition

Product type Book

Published in Apr 2024

Publisher Packt

ISBN-13 9781835080115

Pages 476 pages

Edition 2nd Edition

Languages

Concepts

Data Engineering

Author (1):

Adi Wijaya

Table of Contents (19) Chapters

Preface

1. Part 1: Getting Started with Data Engineering with GCP

2. Chapter 1: Fundamentals of Data Engineering

3. Chapter 2: Big Data Capabilities on GCP

4. Part 2: Build Solutions with GCP Components

5. Chapter 3: Building a Data Warehouse in BigQuery

6. Chapter 4: Building Workflows for Batch Data Loading Using Cloud Composer

7. Chapter 5: Building a Data Lake Using Dataproc

8. Chapter 6: Processing Streaming Data with Pub/Sub and Dataflow

9. Chapter 7: Visualizing Data to Make Data-Driven Decisions with Looker Studio

10. Chapter 8: Building Machine Learning Solutions on GCP

11. Part 3: Key Strategies for Architecting Top-Notch Solutions

12. Chapter 9: User and Project Management in GCP

13. Chapter 10: Data Governance in GCP

14. Chapter 11: Cost Strategy in GCP

15. Chapter 12: CI/CD on GCP for Data Engineers

16. Chapter 13: Boosting Your Confidence as a Data Engineer

17. Index

Why subscribe?

18. Other Books You May Enjoy

Going through the foundational concepts for data engineering

Even though there are many data engineering concepts that we will learn throughout the book by using Google Cloud Platform (GCP), there are some basic concepts that you need to know as data engineers. In my experience of interviewing in data companies, I discovered that these foundational concepts are often asked to assess how much you know about data engineering. Take the following examples:

What is ETL?
What’s the difference between ETL and Extract, Load, and Transform (ELT)?
What is big data?
How do you handle large volumes of data?

These questions are quite common, yet particularly important to deeply understand the concepts since they may affect our decisions on architecting our data life cycles.

ETL concept in data engineering

ETL is the key foundation of data engineering. Everything in the data life cycle is ETL; any part that happens from upstream to downstream is ETL. Let&...