Reader small image

You're reading from  Cloud Scale Analytics with Azure Data Services

Product typeBook
Published inJul 2021
PublisherPackt
ISBN-139781800562936
Edition1st Edition
Right arrow
Author (1)
Patrik Borosch
Patrik Borosch
author image
Patrik Borosch

Patrik Borosch is a cloud solution architect for data and AI at Microsoft Switzerland GmbH. He has more than 25 years of BI and analytics development, engineering, and architecture experience and is a Microsoft Certified Data Engineer and a Microsoft Certified AI Engineer. Patrik has worked on numerous significant international data warehouse, data integration, and big data projects. Through this, he has built and extended his experience in all facets, from requirements engineering to data modeling and ETL, all the way to reporting and dashboarding. At Microsoft Switzerland, he supports customers in their journey into the analytical world of the Azure Cloud.
Read more about Patrik Borosch

Right arrow

Talking about partitioning

When you need to load massive amounts of data to your database, partitioning might be another optimization option. But you really should be confronted with massive amounts of data when you start considering partitioning.

Do you remember the math of the CCI and why it will only perform when you load around 63 to 100 million rows to your database (see the preceding section, Understanding CCI)? Now, you need to add another factor to this equation: the number of partitions that you are planning for your database.

Let's assume that you want to have one partition for every month (the most typical usage of partitions) in your table, and you plan to load data for 5 years to your database. This will add another 60 as a factor to your preceding term: 60 distributions x 1,048,578 rows per distribution x 60 months in the database. This results in 3,774,880,800 rows that your table needs to hold as a minimum in order for the CCI to be built over all the partitions...

lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Cloud Scale Analytics with Azure Data Services
Published in: Jul 2021Publisher: PacktISBN-13: 9781800562936

Author (1)

author image
Patrik Borosch

Patrik Borosch is a cloud solution architect for data and AI at Microsoft Switzerland GmbH. He has more than 25 years of BI and analytics development, engineering, and architecture experience and is a Microsoft Certified Data Engineer and a Microsoft Certified AI Engineer. Patrik has worked on numerous significant international data warehouse, data integration, and big data projects. Through this, he has built and extended his experience in all facets, from requirements engineering to data modeling and ETL, all the way to reporting and dashboarding. At Microsoft Switzerland, he supports customers in their journey into the analytical world of the Azure Cloud.
Read more about Patrik Borosch