You're reading from Modern Data Architecture on AWS

Product typeBook

Published inAug 2023

PublisherPackt

ISBN-139781801813396

Edition1st Edition

Concepts

Data Science

Author (1)

Behram Irani

Interactive Analytics

In this chapter, we will look at the following key topics:

Analytics using Amazon Athena
Analytics using Presto, Trino, and Hive on Amazon EMR

One of the fundamental principles of building a modern data architecture on AWS is hinged around using purpose-built tools for solving specific use cases. An enterprise data platform once fully built has many components, each with a specific purpose for solving a particular business use case.

In Chapter 2, Scalable Data Lakes, we went through the fundamentals of building a data lake on AWS using Amazon S3 as the storage layer and the AWS Glue Data Catalog as the technical metadata layer. Each layer of the data lake has data that may be of use to different personas in an organization. The most basic ask from each of these personas will be to provide them the ability to query datasets in the data lake using the SQL syntax so that they can derive insights from the data. Interactive analytics, using specific...

Analytics using Amazon Athena

Amazon Athena is a serverless service that allows anyone to interactively query data in an S3 data lake using American National Standards Institute (ANSI) SQL. Athena is integrated with the Glue Data Catalog, which means that as soon as the metadata about the data is captured in the catalog, Athena is able to view and query those tables without any additional setup.

With no infrastructure to manage, along with direct out-of-the-box integration with Glue Data Catalog, this makes Athena a very popular service for multiple personas in the organization for interactive and ad hoc analytics. And being a serverless service, the pricing model of the service, 5 dollars per terabyte of data scanned, is also very appealing to organizations. With the correct optimization techniques, you can effectively roll out Athena as the service of choice throughout your organization for all types of interactive analytics use cases, for querying the data stored in the S3 data...

Analytics using Presto, Trino, and Hive on Amazon EMR

If you recall from Chapter 5, we introduced Amazon EMR as one of the services for processing big data. EMR has over 25 open source projects, and we went through a use case where Apache Spark in EMR was leveraged to solve a data processing problem. EMR also has a few projects that assist in ad hoc query execution and allow users to interactively executive SQL queries to get the data stored in the S3 data lake. Let’s shed some light on these projects.

Presto/Trino

Presto is an open source project that provides a fast analytics query execution engine for data stored in many types of storage, most commonly used with data stored in data lakes. Presto, also known as PrestoDB, was first created on Facebook. In 2019, Presto development eventually forked into two, with PrestoDB and PrestoSQL. To keep the name confusion at a minimum, PrestoSQL was renamed Trino in 2020.

Amazon EMR supports both PrestoDB and Trino. Organizations...

Summary

In this chapter, we looked at how Amazon Athena and Presto, Trino, and Hive on EMR help organizations perform ad hoc interactive data analytics on the data stored in the S3 data lake. Athena is a serverless platform that integrates with the Glue Data Catalog and provides data analysts with the ability to write and execute SQL queries without having to manage the platform itself. Using Athena, organizations can focus on the business logic needed for reports versus spending time on creating and managing the infrastructure that’s required by the platform.

We also looked at cases when creating a Presto/Trino cluster on Amazon EMR may be more beneficial for interactive analytics. This is particularly helpful when there are very large volumes of datasets that need to be scanned by thousands of queries on a daily basis and where performance SLAs are strict. Using Presto/Trino on EMR, customers can control cost and at the same time improve query performance by custom tuning...

References

Amazon Athena workshop: https://catalog.us-east-1.prod.workshops.aws/workshops/9981f1a1-abdc-49b5-8387-cb01d238bb78/en-US

The rest of the chapter is locked

You have been reading a chapter from

Modern Data Architecture on AWS

Published in: Aug 2023Publisher: PacktISBN-13: 9781801813396

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

undefined

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at $15.99/month. Cancel anytime

Author (1)

Behram Irani

Behram Irani is currently a technology leader with Amazon Web Services (AWS) specializing in data, analytics and AI/ML. He has spent over 18 years in the tech industry helping organizations, from start-ups to large-scale enterprises, modernize their data platforms. In the last 6 years working at AWS, Behram has been a thought leader in the data, analytics and AI/ML space; publishing multiple papers and leading the digital transformation efforts for many organizations across the globe. Behram has completed his Bachelor of Engineering in Computer Science from the University of Pune and has an MBA degree from the University of Florida.
Read more about Behram Irani

Personalised recommendations for you

Based on your interests and search pattern

Et al.

Ever wonder why speech recognition systems don't understand the Scottish accent, or what would happen if an astronaut only ate mac 'n' cheese, or other spurious reflections you'd have at a bar? We did, then collated those deliberations into absurd research articles with fake figures and methodologies inspired by even more fictionally absurd studies.

BookAug 2023230 pages5

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages4

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages5

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages1

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages5

Mastering Tableau 2023

This book is a comprehensive resource to mastering your Tableau skills and becoming a BI expert. As you progress, you will learn how to build advanced dashboards and improve your storytelling to derive key business insight, as well as make you well-versed with advanced functionalities of Tableau in the business intelligence domain.

BookAug 2023684 pages

Building AI Applications with ChatGPT APIs

This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.

BookSep 2023258 pages5

Building AI Applications with ChatGPT APIs

This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.

BookSep 2023258 pages2

Data Engineering with AWS

Embark on a journey to master data engineering pipelines on AWS! Our book offers a hands-on experience of AWS services for ingesting, transforming, and consuming data. Whether you're an absolute beginner or someone with basic data engineering experience, this guide is an indispensable resource.

BookOct 2023636 pages5

Modern Data Architecture on AWS

Every organization wants an agile, performant, and cost-effective data platform that meets all their current and future business needs. Purpose-built AWS analytics services and their features play a big part in building such a modern data platform. This book brings to you all the design and architectural patterns that’ll help you achieve this goal.

BookAug 2023420 pages5

Practical Guide to Applied Conformal Prediction in Python

Discover the power of Conformal Prediction with the "Practical Guide to Applied Conformal Prediction in Python." Master the latest techniques to quantify uncertainty in machine learning and computer vision models, and seamlessly apply them to your industry applications.

BookDec 2023240 pages

TinyML Cookbook

With over 70 project-based recipes, the TinyML Cookbook is a practical guide that will help you to get the most out of your microcontrollers. It provides a comprehensive understanding of the theoretical foundations while giving you hands-on experience training ML models for deployment on Arduino Nano 33 BLE Sense, Raspberry Pi Pico, and SparkFun RedBoard Artemis Nano microcontrollers.

BookNov 2023664 pages