Mastering Apache Spark 2.x - Second Edition

Advanced analytics on your Big Data with latest Apache Spark 2.x
Preview in Mapt

Mastering Apache Spark 2.x - Second Edition

Romeo Kienzler

1 customer reviews
Advanced analytics on your Big Data with latest Apache Spark 2.x

Quick links: > What will you learn?> Table of content> Product reviews

eBook
$28.00
RRP $39.99
Save 29%
Print + eBook
$49.99
RRP $49.99
What do I get with a Mapt Pro subscription?
  • Unlimited access to all Packt’s 5,000+ eBooks and Videos
  • Early Access content, Progress Tracking, and Assessments
  • 1 Free eBook or Video to download and keep every month after trial
What do I get with an eBook?
  • Download this book in EPUB, PDF, MOBI formats
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
What do I get with Print & eBook?
  • Get a paperback copy of the book delivered to you
  • Download this book in EPUB, PDF, MOBI formats
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
What do I get with a Video?
  • Download this Video course in MP4 format
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
$28.00
$49.99
RRP $39.99
RRP $49.99
eBook
Print + eBook

Frequently bought together


Mastering Apache Spark 2.x - Second Edition Book Cover
Mastering Apache Spark 2.x - Second Edition
$ 39.99
$ 28.00
Apache Spark 2.x Cookbook Book Cover
Apache Spark 2.x Cookbook
$ 39.99
$ 28.00
Buy 2 for $35.00
Save $44.98
Add to Cart

Book Details

ISBN 139781786462749
Paperback354 pages

Book Description

Apache Spark is an in-memory cluster-based parallel processing system that provides a wide range of functionalities such as graph processing, machine learning, stream processing, and SQL. This book aims to take your knowledge of Spark to the next level by teaching you how to expand Spark’s functionality and implement your data flows and machine/deep learning programs on top of the platform.

The book commences with an overview of the Spark ecosystem. It will introduce you to Project Tungsten and Catalyst, two of the major advancements of Apache Spark 2.x.

You will understand how memory management and binary processing, cache-aware computation, and code generation are used to speed things up dramatically. The book extends to show how to incorporate H20, SystemML, and Deeplearning4j for machine learning, and Jupyter Notebooks and Kubernetes/Docker for cloud-based Spark. During the course of the book, you will learn about the latest enhancements to Apache Spark 2.x, such as interactive querying of live data and unifying DataFrames and Datasets.

You will also learn about the updates on the APIs and how DataFrames and Datasets affect SQL, machine learning, graph processing, and streaming. You will learn to use Spark as a big data operating system, understand how to implement advanced analytics on the new APIs, and explore how easy it is to use Spark in day-to-day tasks.

Table of Contents

Chapter 1: A First Taste and What’s New in Apache Spark V2
Spark machine learning
Spark Streaming
Spark SQL
Spark graph processing
Extended ecosystem
What's new in Apache Spark V2?
Cluster design
Cluster management
Cloud-based deployments
Performance
Cloud
Summary
Chapter 2: Apache Spark SQL
The SparkSession--your gateway to structured data processing
Importing and saving data
Understanding the DataSource API
DataFrames
Using SQL
Using Datasets
User-defined functions
RDDs versus DataFrames versus Datasets
Summary
Chapter 3: The Catalyst Optimizer
Understanding the workings of the Catalyst Optimizer
Managing temporary views with the catalog API
The SQL abstract syntax tree
How to go from Unresolved Logical Execution Plan to Resolved Logical Execution Plan
Code generation
Summary
Chapter 4: Project Tungsten
Memory management beyond the Java Virtual Machine Garbage Collector
Cache-friendly layout of data in memory
Code generation
Summary
Chapter 5: Apache Spark Streaming
Overview
Errors and recovery
Streaming sources
Summary
Chapter 6: Structured Streaming
The concept of continuous applications
Windowing
Increased performance with good old friends
How transparent fault tolerance and exactly-once delivery guarantee is achieved
Example - connection to a MQTT message broker
Summary
Chapter 7: Apache Spark MLlib
Architecture
Classification with Naive Bayes
Clustering with K-Means
Artificial neural networks
Summary
Chapter 8: Apache SparkML
What does the new API look like?
The concept of pipelines
Model evaluation
CrossValidation and hyperparameter tuning
Winning a Kaggle competition with Apache SparkML
Summary
Chapter 9: Apache SystemML
Why do we need just another library?
A cost-based optimizer for machine learning algorithms
Performance measurements
Apache SystemML in action
Summary
Chapter 10: Deep Learning on Apache Spark with DeepLearning4j and H2O
H2O
Deeplearning4j
Summary
Chapter 11: Apache Spark GraphX
Overview
Graph analytics/processing with GraphX
Summary
Chapter 12: Apache Spark GraphFrames
Architecture
Examples
Summary
Chapter 13: Apache Spark with Jupyter Notebooks on IBM DataScience Experience
Why notebooks are the new standard
Learning by example
Summary
Chapter 14: Apache Spark on Kubernetes
Bare metal, virtual machines, and containers
Understanding the core concepts of Docker
Understanding Kubernetes
Using Kubernetes for provisioning containerized Spark applications
Example--Apache Spark on Kubernetes
Summary

What You Will Learn

  • Examine Advanced Machine Learning and DeepLearning with MLlib, SparkML, SystemML, H2O and DeepLearning4J
  • Study highly optimised unified batch and real-time data processing using SparkSQL and Structured Streaming
  • Evaluate large-scale Graph Processing and Analysis using GraphX and GraphFrames
  • Apply Apache Spark in Elastic deployments using Jupyter and Zeppelin Notebooks, Docker, Kubernetes and the IBM Cloud
  • Understand internal details of cost based optimizers used in Catalyst, SystemML and GraphFrames
  • Learn how specific parameter settings affect overall performance of an Apache Spark cluster
  • Leverage Scala, R and python for your data science projects

Authors

Table of Contents

Chapter 1: A First Taste and What’s New in Apache Spark V2
Spark machine learning
Spark Streaming
Spark SQL
Spark graph processing
Extended ecosystem
What's new in Apache Spark V2?
Cluster design
Cluster management
Cloud-based deployments
Performance
Cloud
Summary
Chapter 2: Apache Spark SQL
The SparkSession--your gateway to structured data processing
Importing and saving data
Understanding the DataSource API
DataFrames
Using SQL
Using Datasets
User-defined functions
RDDs versus DataFrames versus Datasets
Summary
Chapter 3: The Catalyst Optimizer
Understanding the workings of the Catalyst Optimizer
Managing temporary views with the catalog API
The SQL abstract syntax tree
How to go from Unresolved Logical Execution Plan to Resolved Logical Execution Plan
Code generation
Summary
Chapter 4: Project Tungsten
Memory management beyond the Java Virtual Machine Garbage Collector
Cache-friendly layout of data in memory
Code generation
Summary
Chapter 5: Apache Spark Streaming
Overview
Errors and recovery
Streaming sources
Summary
Chapter 6: Structured Streaming
The concept of continuous applications
Windowing
Increased performance with good old friends
How transparent fault tolerance and exactly-once delivery guarantee is achieved
Example - connection to a MQTT message broker
Summary
Chapter 7: Apache Spark MLlib
Architecture
Classification with Naive Bayes
Clustering with K-Means
Artificial neural networks
Summary
Chapter 8: Apache SparkML
What does the new API look like?
The concept of pipelines
Model evaluation
CrossValidation and hyperparameter tuning
Winning a Kaggle competition with Apache SparkML
Summary
Chapter 9: Apache SystemML
Why do we need just another library?
A cost-based optimizer for machine learning algorithms
Performance measurements
Apache SystemML in action
Summary
Chapter 10: Deep Learning on Apache Spark with DeepLearning4j and H2O
H2O
Deeplearning4j
Summary
Chapter 11: Apache Spark GraphX
Overview
Graph analytics/processing with GraphX
Summary
Chapter 12: Apache Spark GraphFrames
Architecture
Examples
Summary
Chapter 13: Apache Spark with Jupyter Notebooks on IBM DataScience Experience
Why notebooks are the new standard
Learning by example
Summary
Chapter 14: Apache Spark on Kubernetes
Bare metal, virtual machines, and containers
Understanding the core concepts of Docker
Understanding Kubernetes
Using Kubernetes for provisioning containerized Spark applications
Example--Apache Spark on Kubernetes
Summary

Book Details

ISBN 139781786462749
Paperback354 pages
Read More
From 1 reviews

Read More Reviews

Recommended for You

Apache Spark 2.x Cookbook Book Cover
Apache Spark 2.x Cookbook
$ 39.99
$ 28.00
Learning PySpark Book Cover
Learning PySpark
$ 35.99
$ 25.20
Apache Spark 2.x for Java Developers Book Cover
Apache Spark 2.x for Java Developers
$ 39.99
$ 28.00
Building Data Streaming Applications with Apache Kafka Book Cover
Building Data Streaming Applications with Apache Kafka
$ 35.99
$ 25.20
Frank Kane's Taming Big Data with Apache Spark and Python Book Cover
Frank Kane's Taming Big Data with Apache Spark and Python
$ 31.99
$ 22.40
Data Visualization with D3 4.x Cookbook - Second Edition Book Cover
Data Visualization with D3 4.x Cookbook - Second Edition
$ 35.99
$ 25.20