Reader small image

You're reading from  AWS Certified Machine Learning Specialty: MLS-C01 Certification Guide

Product typeBook
Published inMar 2021
Reading LevelBeginner
PublisherPackt
ISBN-139781800569003
Edition1st Edition
Languages
Right arrow
Authors (2):
Somanath Nanda
Somanath Nanda
author image
Somanath Nanda

Somanath has 10 years of working experience in IT industry which includes Prod development, Devops, Design and architect products from end to end. He has also worked at AWS as a Big Data Engineer for about 2 years.
Read more about Somanath Nanda

Weslley Moura
Weslley Moura
author image
Weslley Moura

Weslley Moura has been developing data products for the past decade. At his recent roles, he has been influencing data strategy and leading data teams into the urban logistics and blockchain industries.
Read more about Weslley Moura

View More author details
Right arrow

Creating ETL jobs on AWS Glue

In a modern data pipeline, there are multiple stages, such as Generate Data, Collect Data, Store Data, Perform ETL, Analyze, and Visualize. In this section, we will cover each of these at a high level and understand the ETL (extract, transform, load) part in-depth:

  • Data can be generated from several devices, including mobile devices or IoT, weblogs, social media, transactional data, online games, and many more besides.
  • This huge amount of generated data can be collected by using polling services or through API gateways integrated with AWS Lambda to collect the data, or via streams such as AWS Kinesis or AWS-managed Kafka or Kinesis Firehose. If you have an on-premises database and you want to collect that data to AWS, then you choose AWS DMS for that. You can sync your on-premises data to Amazon S3, Amazon EFS, or Amazon FSx via AWS DataSync. AWS Snowball is used to collect/transfer data into and out of AWS.
  • The next step involves storing...
lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
AWS Certified Machine Learning Specialty: MLS-C01 Certification Guide
Published in: Mar 2021Publisher: PacktISBN-13: 9781800569003

Authors (2)

author image
Somanath Nanda

Somanath has 10 years of working experience in IT industry which includes Prod development, Devops, Design and architect products from end to end. He has also worked at AWS as a Big Data Engineer for about 2 years.
Read more about Somanath Nanda

author image
Weslley Moura

Weslley Moura has been developing data products for the past decade. At his recent roles, he has been influencing data strategy and leading data teams into the urban logistics and blockchain industries.
Read more about Weslley Moura