Data Modeling

Data modeling is a proactive design process that establishes relationships between data within information systems. A well-designed data model can lower costs, contribute to data democratization, and adapt easily to changing needs.

This chapter explores the various data modeling techniques and what to consider when building a data model. In particular, it focuses on the following:

• The importance of data models
• Designing your data model
• Data modeling techniques
• Choosing a data model

The importance of data models

The primary objective of an effective data model is to translate business requirements into data structures that facilitate the understanding of the current status, identify new opportunities, and drive initiatives based on insights gained from data analysis.

For analytics engineers, it is crucial to build data systems with long-term planning in mind, ensuring seamless integration between business requirements and the data procured in the system. As a result, data modeling is one of the most vital tools to prevent future issues and maintain optimal system performance as complexity increases.

In traditional data warehouses, data models were essential to manage storage and computational costs. However, as seen in Chapter 4, the shift to cloud computing presents another set of challenges.

With the increasing adoption of cloud computing, many of the problems data modeling solves can now be overcome by harnessing the increased computational power of...

Designing your data model

The design process of a data model progresses from generic to specific. The first step is the conceptual diagram, which provides a high-level, generic view of how the data will be stored and connected within the information system and its relationship with business activities. It is usually technology-agnostic. With this initial diagram, the modeler seeks informed feedback from business specialists, data owners, and end users to ensure that the model accurately represents the business and that the main data relationships are represented.

Figure 5.2 shows an example of a conceptual diagram for a simple management system. This diagram contains the basic building blocks of the data model to start the modeling process. We can see how the elements are related. For example, one order item represents a product and is part of an order; the customer places an order, and the model generates an invoice with invoice items; each invoice item represents a product:

...

Data modeling techniques

During the 1990s, as data warehouses rapidly integrated into numerous enterprises and businesses, Bill Inmon, Ralph Kimball, and Daniel Linstedt developed methodologies with architectures for constructing data warehouses.

Each of these methods has its own associated data model. While other methods or possible combinations exist, the three main forms discussed in the following subsections are the most common.

Bill Inmon and relational modeling

In 1992, William (Bill) Inmon published Building the Data Warehouse (https://www.wiley.com/en-us/Building+the+Data+Warehouse,+4th+Edition-p-9780764599446). For this work, he is recognized as one of the fathers of the data warehouse.

Inmon’s methodology for data warehouse architecture highlights the importance of a unified data storage system structured according to the third normal form (3NF), which will be described in depth in the next section. He asserts that robust relational modeling contributes...

Choosing a data model

Modelers should be familiar with these modeling techniques and apply them when appropriate. In practice, most systems have mixed data models, and finding a pure data model that adheres to all the recommended characteristics explored in this chapter is nearly impossible.

As a general guideline, normalization techniques are suitable for transactional systems focused on capturing events, sometimes involving millions of rows in a short period. The strength of the 3NF is that the updates and inserts of these transactions impact the database in only one place.

The dimensional model or star schema is more suitable for analytical purposes, such as drilling down on data, generating reports, and performing aggregations or calculations. This modeling technique is integrated into some business intelligence tools, making it easier for users to understand and query directly for data extraction and create reports or dashboards.

Highly complex systems with multiple sources...

Summary

In this chapter, we have covered the benefits of data modeling, some of the best practices to build a data model, and the principal methodologies used to model a data warehouse. After reviewing their main characteristics, we covered how to choose the best model to fulfill the needs of the business according to the use case.

In the next chapter, we will talk about data transformation and how the evolution of tools has drastically changed the way it is carried out.