Reader small image

You're reading from  Azure Data Engineer Associate Certification Guide

Product typeBook
Published inFeb 2022
PublisherPackt
ISBN-139781801816069
Edition1st Edition
Tools
Concepts
Right arrow
Author (1)
Newton Alex
Newton Alex
author image
Newton Alex

Newton Alex leads several Azure Data Analytics teams in Microsoft, India. His team contributes to technologies including Azure Synapse, Azure Databricks, Azure HDInsight, and many open source technologies, including Apache YARN, Apache Spark, and Apache Hive. He started using Hadoop while at Yahoo, USA, where he helped build the first batch processing pipelines for Yahoo's ad serving team. After Yahoo, he became the leader of the big data team at Pivotal Inc., USA, where he was responsible for the entire open source stack of Pivotal Inc. He later moved to Microsoft and started the Azure Data team in India. He has worked with several Fortune 500 companies to help build their data systems on Azure.
Read more about Newton Alex

Right arrow

Normalizing and denormalizing values

We have already seen the ADF Flatten activity, which helps to denormalize data. There are two more such transformations to help normalize and denormalize datasets—Pivot and Unpivot. Let's look at them in detail.

Denormalizing values using Pivot

Let's assume that you have a table with a normalized column to store City values, but for reporting purposes, you want to have one column per city in your tables. In such a case, you can use the Pivot function to denormalize the table. The Pivot function takes the unique row values and converts them into table columns. Here is an example of how to pivot the tables:

  1. Let's consider the following sample table:

Figure 8.32 – Sample table before pivoting

  1. Select the Pivot activity from the ADF Data Flow tile, and in the Group by tab, specify Gender, as illustrated in the following screenshot:

Figure 8.33 –...

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Azure Data Engineer Associate Certification Guide
Published in: Feb 2022Publisher: PacktISBN-13: 9781801816069

Author (1)

author image
Newton Alex

Newton Alex leads several Azure Data Analytics teams in Microsoft, India. His team contributes to technologies including Azure Synapse, Azure Databricks, Azure HDInsight, and many open source technologies, including Apache YARN, Apache Spark, and Apache Hive. He started using Hadoop while at Yahoo, USA, where he helped build the first batch processing pipelines for Yahoo's ad serving team. After Yahoo, he became the leader of the big data team at Pivotal Inc., USA, where he was responsible for the entire open source stack of Pivotal Inc. He later moved to Microsoft and started the Azure Data team in India. He has worked with several Fortune 500 companies to help build their data systems on Azure.
Read more about Newton Alex