Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Serverless ETL and Analytics with AWS Glue

You're reading from  Serverless ETL and Analytics with AWS Glue

Product type Book
Published in Aug 2022
Publisher Packt
ISBN-13 9781800564985
Pages 434 pages
Edition 1st Edition
Languages
Authors (6):
Vishal Pathak Vishal Pathak
Profile icon Vishal Pathak
Subramanya Vajiraya Subramanya Vajiraya
Profile icon Subramanya Vajiraya
Noritaka Sekiyama Noritaka Sekiyama
Profile icon Noritaka Sekiyama
Tomohiro Tanaka Tomohiro Tanaka
Profile icon Tomohiro Tanaka
Albert Quiroga Albert Quiroga
Profile icon Albert Quiroga
Ishan Gaur Ishan Gaur
Profile icon Ishan Gaur
View More author details

Table of Contents (20) Chapters

Preface Section 1 – Introduction, Concepts, and the Basics of AWS Glue
Chapter 1: Data Management – Introduction and Concepts Chapter 2: Introduction to Important AWS Glue Features Chapter 3: Data Ingestion Section 2 – Data Preparation, Management, and Security
Chapter 4: Data Preparation Chapter 5: Data Layouts Chapter 6: Data Management Chapter 7: Metadata Management Chapter 8: Data Security Chapter 9: Data Sharing Chapter 10: Data Pipeline Management Section 3 – Tuning, Monitoring, Data Lake Common Scenarios, and Interesting Edge Cases
Chapter 11: Monitoring Chapter 12: Tuning, Debugging, and Troubleshooting Chapter 13: Data Analysis Chapter 14: Machine Learning Integration Chapter 15: Architecting Data Lakes for Real-World Scenarios and Edge Cases Other Books You May Enjoy

Optimizing your storage with Amazon S3

So far, we’ve seen how we should store data optimally and how we can manage data to optimize data retrieval and accelerate the analytic workloads. The techniques primarily work on the data itself, such as storing data with columnar formats, data compaction, and more. Not only does it handle data itself optimally, but it’s also important to think about optimization on the storage side. 

Our data, such as logs of web access, device data, and so on, is continuously reported, and that data size grows over time. As the storage usage increases, the cost increases, too. To reduce the cost of storage usage, usually, we archive data that is not frequently or ever accessed. Generally, we can divide data into the following tiers based on the frequency of access to it:

  • Hot: This is data that you usually access.
  • Warm: This is data that you have relatively less access to or require less than hot data.
  • Cold: This is data...
lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime}