Preface

This book was written by a team of seven analytics engineers from Xebia, an international consultancy company in the Netherlands. As Xebians, knowledge sharing is a key part of our culture. In addition to helping our clients, we organize biweekly knowledge exchange sessions and monthly innovation days. As a result, our colleagues are always actively contributing to open-source software, organizing meetups, speaking at data conferences, and writing books.

While we are colleagues at the same company, each of us brings unique expertise and skills to the table. Analytics engineering is a diverse field, and mastering every aspect is not expected. In this book, each author contributed two or three chapters based on their interests and expertise. The same applies to you as a reader. Depending on your organization’s needs, certain chapters may hold greater relevance. Our goal is to provide a comprehensive overview of essential concepts, tailored to the varied demands of this profession.

Since its inception in 2019, analytics engineering has experienced remarkable growth. As its importance continues to be recognized by organizations worldwide, we firmly believe that analytics engineers will become indispensable members of every data team. Through this book, you have the opportunity to become a frontrunner in this evolving field, contributing to its advancement and shaping its future alongside us.

Who this book is for

This book is for data engineers and analysts considering transitioning to analytics engineering, as well as for analytics engineers looking to improve their skills and identify areas for growth.

While introductory in nature, you will benefit from having prior knowledge in key areas such as data analysis processes, including data ingestion, transformation, and visualization, along with database fundamentals such as data modeling, querying, database management, and architecture.

What this book covers

Chapter 1, What Is Analytics Engineering?, traces the history of analytics engineering and its surge in popularity. Dive into its why and what to understand role responsibilities thoroughly.

Chapter 2, The Modern Data Stack, explores the modern data stack, demystifying SQL and cloud impact. Witness the industry’s shift to purpose-built tools for contemporary data management.

Chapter 3, Data Ingestion, explores fundamental techniques, common issues, and strategies for moving data between systems. We break down data ingestion into eight steps and elaborate on common considerations for data quality and scalability.

Chapter 4, Data Warehousing, delves into the core concepts and history of data warehouses. You will gain insights into the evolution of data storage solutions and the impact of cloud technologies on this space.

Chapter 5, Data Modeling, details the proactive design process of establishing relationships between data within information systems. This critical process can reduce costs, enhance computational speed, and elevate user experience.

Chapter 6, Transforming Data, unravels the shift from ETL to ELT pipeline paradigms, the importance of data cleaning and transformation, reusability in query processes, and optimizing SQL queries for modularity.

Chapter 7, Serving Data, discusses presenting data to end users. You will gain insights into exposing data through different means, understanding data as a product, and examining the motivations and challenges associated with achieving self-service analytics in companies.

Chapter 8, Hands-On Analytics Engineering, also describes tools such as Airbyte Cloud for managed ingestion, Google BigQuery for warehousing, dbt Cloud for transformations, and Tableau for visualization.

Chapter 9, Data Quality and Observability, helps you ensure data quality and establish observability in analytics processes. Delving into strategies and tools, this chapter equips you with the skills to maintain data integrity and transparency.

Chapter 10, Writing Code in a Team, focuses on collaborative coding practices within a team setting. With an emphasis on best practices, version control, and effective communication, this chapter focuses on teamwork and efficiency in analytics engineering projects.

Chapter 11, Automating Workflows, concludes the DataOps section by exploring the implementation of continuous workflows. You will be introduced to practices that streamline analytics workflows, optimizing efficiency and productivity.

Chapter 12, Driving Business Adoption, highlights the critical process of collecting and interpreting business requirements. You will be guided through the steps to understand and align analytics initiatives with the unique needs and objectives of the business.

Chapter 13, Data Governance, delves into the principles and practices of data governance. You will gain insights into establishing robust data governance frameworks, ensuring the reliability of organizational data, and aligning analytics strategies with overarching business goals.

Chapter 14, Epilogue, summarizes the learning from this book and gives you extra tips to take your analytics engineering career even further.

To get the most out of this book

Familiarity with Python and SQL will enhance comprehension of Chapters 8, 10, and 11. Additionally, basic knowledge of command-line usage (with tools such as Git and dbt) and familiarity with cloud computing concepts will be beneficial, as this book serves as an introduction to these topics.

The following software/hardware are covered in the book:

Software/Hardware are covered in the book	Operating system requirement
Airbyte Cloud	WindowsOS, macOS, or Linux
dbt Cloud
Google Cloud, Google BigQuery, Google Sheets
Tableau Desktop
Git and GitHub

Some of these tools have paid versions. To follow the instructions in this book, you can use the free versions or make use of a free trial.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Several chapters in the book feature code snippets designed to showcase best practices. However, executing these snippets may need extra setup, not covered in this book. You should view these snippets as illustrative examples and adapt the underlying best practices to your unique scenarios.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Fundamentals-of-Analytics-Engineering. If there’s an update to the code, it will be updated in the GitHub repository.

There are several hands-on guides mentioned in the book, they can be found at https://github.com/PacktPublishing/Fundamentals-of-Analytics-Engineering/tree/main/chapter_8/guides.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Notice how the next CTE, called employees, selects from the raw_source CTE.”

A block of code is set as follows:

def add_numbers(a, b):
    c = a + b
    return c

Any command-line input or output is written as follows:

on-run-end: "{{ dbt_project_evaluator.print_dbt_project_evaluator_issues() }}"

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “A common application for the ETL process is when organizations have strict requirements regarding Personal Identifiable Information (PII).”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [e-mail] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [e-mail] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Fundamentals of Analytics Engineering, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

1. Scan the QR code or visit the link below

https://packt.link/free-ebook/9781837636457

2. Submit your proof of purchase
3. That’s it! We’ll send your free PDF and other benefits to your email directly