Part 3: Key Strategies for Architecting Top-Notch Solutions

In the final part of this book, we will cover topics beyond how to use the GCP products for building data pipelines. A good data engineer not only knows how to develop a solution but also decides important aspects outside the solution itself – for example, creating a GCP project is easy, but how many projects should you create? What is the purpose of the project? Should you assign one or many users to the project? Those are a few of many important questions to be considered and decided in order to build top-notch solutions for organizations.

This will include what the thinking process is on how to choose a user and project structure, understanding the implementation of data governance, how to estimate the cost for the solutions, and how to make sure the development can be tested and deployed automatically using CI/CD concepts. After all the learning chapters, you should have gained much more technical knowledge,...