Data Engineering Vol2 AWS Data Processing - Spark & Kafka: Build batch and streaming data pipelines with Spark, Kafka, AWS EMR, MSK, and Iceberg

Soumyadeep Dey

€53.99

Video Jul 2026 47hrs 27mins 1st Edition

Soumyadeep Dey

€53.99

Video Jul 2026 47hrs 27mins 1st Edition

What do you get with a video?

Download this video in MP4 format

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

View table of contents

Download Code

Key benefits

Spark and Kafka coverage spans batch, stream, SQL, partitioning, tuning, and lakehouse use
AWS-based practice connects EMR, MSK, S3, RDS, Redshift, Lambda, and Iceberg workflows at scale
Large-dataset projects mirror real processing, ingestion, optimization, and analytics scenarios

Description

Modern data engineering depends on reliable batch and streaming systems that can process large datasets, support analytics, and connect storage with downstream applications. Spark and Kafka are central to that workflow, especially when hosted on AWS services such as EMR, MSK, and S3. The learning path begins with big data processing foundations before moving into Spark architecture, DataFrames, transformations, actions, partitions, shuffling, Spark SQL, joins, functions, tuning, and deployment modes. It also covers Kafka for streaming ingestion, Spark Streaming for stream processing, AWS Lambda for data processing, Databricks, Redshift integration, and data lakehouse concepts using Apache Iceberg. Practical projects and large dataset exercises help learners connect concepts with implementation. The material supports stronger decision-making around pipeline design, performance tuning, batch versus stream processing, and cloud service selection. By the end of this course, learners can build, tune, and explain Spark and Kafka data processing workflows on AWS.

Who is this book for?

Best suited for Python developers, application developers, big data developers, data engineers, data scientists, data analysts, database administrators, architects, technical managers, and data engineering aspirants. AWS and SQL knowledge is helpful.