Scala and Spark for Big Data Analytics

Harness the power of Scala to program Spark and analyze tonnes of data in the blink of an eye!
Preview in Mapt

Scala and Spark for Big Data Analytics

Md. Rezaul Karim, Sridhar Alla

Harness the power of Scala to program Spark and analyze tonnes of data in the blink of an eye!
Mapt Subscription
FREE
$20.83/m after trial
eBook
$36.40
RRP $51.99
Save 29%
Print + eBook
$64.99
RRP $64.99
What do I get with a Mapt Pro subscription?
  • Unlimited access to all Packt’s 5,000+ eBooks and Videos
  • Early Access content, Progress Tracking, and Assessments
  • 1 Free eBook or Video to download and keep every month after trial
What do I get with an eBook?
  • Download this book in EPUB, PDF, MOBI formats
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
What do I get with Print & eBook?
  • Get a paperback copy of the book delivered to you
  • Download this book in EPUB, PDF, MOBI formats
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
What do I get with a Video?
  • Download this Video course in MP4 format
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
$0.00
$36.40
$64.99
$29.99 p/m after trial
RRP $51.99
RRP $64.99
Subscription
eBook
Print + eBook
Start 14 Day Trial

Frequently bought together


Scala and Spark for Big Data Analytics Book Cover
Scala and Spark for Big Data Analytics
$ 51.99
$ 36.40
Basic Statistics and Data Mining for Data Science [Video] Book Cover
Basic Statistics and Data Mining for Data Science [Video]
$ 124.99
$ 106.25
Buy 2 for $35.00
Save $141.98
Add to Cart

Book Details

ISBN 139781785280849
Paperback898 pages

Book Description

Scala has been observing wide adoption over the past few years, especially in the field of data science and analytics. Spark, built on Scala, has gained a lot of recognition and is being used widely in productions. Thus, if you want to leverage the power of Scala and Spark to make sense of big data, this book is for you.

The first part introduces you to Scala, helping you understand the object-oriented and functional programming concepts needed for Spark application development. It then moves on to Spark to cover the basic abstractions using RDD and DataFrame. This will help you develop scalable and fault-tolerant streaming applications by analyzing structured and unstructured data using SparkSQL, GraphX, and Spark structured streaming. Finally, the book moves on to some advanced topics, such as monitoring, configuration, debugging, testing, and deployment.

You will also learn how to develop Spark applications using SparkR and PySpark APIs, interactive data analytics using Zeppelin, and in-memory data processing with Alluxio.

By the end of this book, you will have a thorough understanding of Spark, and you will be able to perform full-stack data analytics with a feel that no amount of data is too big.

Table of Contents

Chapter 1: Introduction to Scala
History and purposes of Scala
Platforms and editors
Installing and setting up Scala
Scala: the scalable language
Scala for Java programmers
Scala for the beginners
Summary
Chapter 2: Object-Oriented Scala
Variables in Scala
Methods, classes, and objects in Scala
Packages and package objects
Java interoperability
Pattern matching
Implicit in Scala
Generic in Scala
SBT and other build systems
Summary
Chapter 3: Functional Programming Concepts
Introduction to functional programming
Functional Scala for the data scientists
Why FP and Scala for learning Spark?
Pure functions and higher-order functions
Using higher-order functions
Error handling in functional Scala
Functional programming and data mutability
Summary
Chapter 4: Collection APIs
Scala collection APIs
Types and hierarchies
Performance characteristics
Java interoperability
Using Scala implicits
Summary
Chapter 5: Tackle Big Data – Spark Comes to the Party
Introduction to data analytics
Introduction to big data
Distributed computing using Apache Hadoop
Here comes Apache Spark
Summary
Chapter 6: Start Working with Spark – REPL and RDDs
Dig deeper into Apache Spark
Apache Spark installation
Introduction to RDDs
Using the Spark shell
Actions and Transformations
Caching
Loading and saving data
Summary
Chapter 7: Special RDD Operations
Types of RDDs
Aggregations
Partitioning and shuffling
Broadcast variables
Accumulators
Summary
Chapter 8: Introduce a Little Structure - Spark SQL
Spark SQL and DataFrames
DataFrame API and SQL API
Aggregations
Joins
Summary
Chapter 9: Stream Me Up, Scotty - Spark Streaming
A Brief introduction to streaming
Spark Streaming
Discretized streams
Stateful/stateless transformations
Checkpointing
Interoperability with streaming platforms (Apache Kafka)
Structured streaming
Summary
Chapter 10: Everything is Connected - GraphX
A brief introduction to graph theory
GraphX
VertexRDD and EdgeRDD
Graph operators
Pregel API
PageRank
Summary
Chapter 11: Learning Machine Learning - Spark MLlib and Spark ML
Introduction to machine learning
Spark machine learning APIs
Feature extraction and transformation
Creating a simple pipeline
Unsupervised machine learning
Binary and multiclass classification
Summary
Chapter 12: My Name is Bayes, Naive Bayes
Multinomial classification
Bayesian inference
Naive Bayes
The decision trees
Summary
Chapter 13: Time to Put Some Order - Cluster Your Data with Spark MLlib
Unsupervised learning
Clustering techniques
Centroid-based clustering (CC)
Hierarchical clustering (HC)
Distribution-based clustering (DC)
Determining number of clusters
A comparative analysis between clustering algorithms
Submitting Spark job for cluster analysis
Summary
Chapter 14: Text Analytics Using Spark ML
Understanding text analytics
Transformers and Estimators
Tokenization
StopWordsRemover
NGrams
TF-IDF
Word2Vec
CountVectorizer
Topic modeling using LDA
Implementing text classification
Summary
Chapter 15: Spark Tuning
Monitoring Spark jobs
Spark configuration
Common mistakes in Spark app development
Optimization techniques
Summary
Chapter 16: Time to Go to ClusterLand - Deploying Spark on a Cluster
Spark architecture in a cluster
Deploying the Spark application on a cluster
Summary
Chapter 17: Testing and Debugging Spark
Testing in a distributed environment
Testing Spark applications
Debugging Spark applications
Summary
Chapter 18: PySpark and SparkR
Introduction to PySpark
Installation and configuration
Introduction to SparkR
Summary

What You Will Learn

  • Understand object-oriented & functional programming concepts of Scala
  • In-depth understanding of Scala collection APIs
  • Work with RDD and DataFrame to learn Spark’s core abstractions
  • Analysing structured and unstructured data using SparkSQL and GraphX
  • Scalable and fault-tolerant streaming application development using Spark structured streaming
  • Learn machine-learning best practices for classification, regression, dimensionality reduction, and recommendation system to build predictive models with widely used algorithms in Spark MLlib & ML
  • Build clustering models to cluster a vast amount of data
  • Understand tuning, debugging, and monitoring Spark applications
  • Deploy Spark applications on real clusters in Standalone, Mesos, and YARN

Authors

Table of Contents

Chapter 1: Introduction to Scala
History and purposes of Scala
Platforms and editors
Installing and setting up Scala
Scala: the scalable language
Scala for Java programmers
Scala for the beginners
Summary
Chapter 2: Object-Oriented Scala
Variables in Scala
Methods, classes, and objects in Scala
Packages and package objects
Java interoperability
Pattern matching
Implicit in Scala
Generic in Scala
SBT and other build systems
Summary
Chapter 3: Functional Programming Concepts
Introduction to functional programming
Functional Scala for the data scientists
Why FP and Scala for learning Spark?
Pure functions and higher-order functions
Using higher-order functions
Error handling in functional Scala
Functional programming and data mutability
Summary
Chapter 4: Collection APIs
Scala collection APIs
Types and hierarchies
Performance characteristics
Java interoperability
Using Scala implicits
Summary
Chapter 5: Tackle Big Data – Spark Comes to the Party
Introduction to data analytics
Introduction to big data
Distributed computing using Apache Hadoop
Here comes Apache Spark
Summary
Chapter 6: Start Working with Spark – REPL and RDDs
Dig deeper into Apache Spark
Apache Spark installation
Introduction to RDDs
Using the Spark shell
Actions and Transformations
Caching
Loading and saving data
Summary
Chapter 7: Special RDD Operations
Types of RDDs
Aggregations
Partitioning and shuffling
Broadcast variables
Accumulators
Summary
Chapter 8: Introduce a Little Structure - Spark SQL
Spark SQL and DataFrames
DataFrame API and SQL API
Aggregations
Joins
Summary
Chapter 9: Stream Me Up, Scotty - Spark Streaming
A Brief introduction to streaming
Spark Streaming
Discretized streams
Stateful/stateless transformations
Checkpointing
Interoperability with streaming platforms (Apache Kafka)
Structured streaming
Summary
Chapter 10: Everything is Connected - GraphX
A brief introduction to graph theory
GraphX
VertexRDD and EdgeRDD
Graph operators
Pregel API
PageRank
Summary
Chapter 11: Learning Machine Learning - Spark MLlib and Spark ML
Introduction to machine learning
Spark machine learning APIs
Feature extraction and transformation
Creating a simple pipeline
Unsupervised machine learning
Binary and multiclass classification
Summary
Chapter 12: My Name is Bayes, Naive Bayes
Multinomial classification
Bayesian inference
Naive Bayes
The decision trees
Summary
Chapter 13: Time to Put Some Order - Cluster Your Data with Spark MLlib
Unsupervised learning
Clustering techniques
Centroid-based clustering (CC)
Hierarchical clustering (HC)
Distribution-based clustering (DC)
Determining number of clusters
A comparative analysis between clustering algorithms
Submitting Spark job for cluster analysis
Summary
Chapter 14: Text Analytics Using Spark ML
Understanding text analytics
Transformers and Estimators
Tokenization
StopWordsRemover
NGrams
TF-IDF
Word2Vec
CountVectorizer
Topic modeling using LDA
Implementing text classification
Summary
Chapter 15: Spark Tuning
Monitoring Spark jobs
Spark configuration
Common mistakes in Spark app development
Optimization techniques
Summary
Chapter 16: Time to Go to ClusterLand - Deploying Spark on a Cluster
Spark architecture in a cluster
Deploying the Spark application on a cluster
Summary
Chapter 17: Testing and Debugging Spark
Testing in a distributed environment
Testing Spark applications
Debugging Spark applications
Summary
Chapter 18: PySpark and SparkR
Introduction to PySpark
Installation and configuration
Introduction to SparkR
Summary

Book Details

ISBN 139781785280849
Paperback898 pages
Read More

Read More Reviews

Recommended for You

Mastering Apache Spark 2.x - Second Edition Book Cover
Mastering Apache Spark 2.x - Second Edition
$ 39.99
$ 28.00
Machine Learning Algorithms Book Cover
Machine Learning Algorithms
$ 39.99
$ 28.00
Statistics for Machine Learning Book Cover
Statistics for Machine Learning
$ 39.99
$ 28.00
Apache Spark 2.x Cookbook Book Cover
Apache Spark 2.x Cookbook
$ 39.99
$ 28.00
Scala: Guide for Data Science Professionals Book Cover
Scala: Guide for Data Science Professionals
$ 71.99
$ 50.40
Scala: Applied Machine Learning Book Cover
Scala: Applied Machine Learning
$ 69.99
$ 49.00