Large Scale Machine Learning with Spark

Discover everything you need to build robust machine learning applications with Spark 2.0

Large Scale Machine Learning with Spark

Learning
Md. Rezaul Karim, Md. Mahedi Kaysar

Discover everything you need to build robust machine learning applications with Spark 2.0
$39.99
$49.99
RRP $39.99
RRP $49.99
eBook
Print + eBook

Instantly access this course right now and get the skills you need in 2017

With unlimited access to a constantly growing library of over 4,000 eBooks and Videos, a subscription to Mapt gives you everything you need to learn new skills. Cancel anytime.

Preview in Mapt

Book Details

ISBN 139781785888748
Paperback476 pages

Book Description

Data processing, implementing related algorithms, tuning, scaling up and finally deploying are some crucial steps in the process of optimising any application.

Spark is capable of handling large-scale batch and streaming data to figure out when to cache data in memory and processing them up to 100 times faster than Hadoop-based MapReduce.This means predictive analytics can be applied to streaming and batch to develop complete machine learning (ML) applications a lot quicker, making Spark an ideal candidate for large data-intensive applications.

This book focuses on design engineering and scalable solutions using ML with Spark. First, you will learn how to install Spark with all new features from the latest Spark 2.0 release. Moving on, you’ll explore important concepts such as advanced feature engineering with RDD and Datasets. After studying developing and deploying applications, you will see how to use external libraries with Spark.

In summary, you will be able to develop complete and personalised ML applications from data collections,model building, tuning, and scaling up to deploying on a cluster or the cloud.

Table of Contents

Chapter 1: Introduction to Data Analytics with Spark
Spark overview
New computing paradigm with Spark
Spark ecosystem
Spark machine learning libraries
Installing and getting started with Spark
Packaging your application with dependencies
Running a sample machine learning application
References
Summary
Chapter 2: Machine Learning Best Practices
What is machine learning?
Machine learning tasks
Practical machine learning problems
Most widely used machine learning problems
Large scale machine learning APIs in Spark
Practical machine learning best practices
Choosing the right algorithm for your application
Summary
Chapter 3: Understanding the Problem by Understanding the Data
Analyzing and preparing your data
Resilient Distributed Dataset basics
Dataset basics
Dataset from string and typed class
Spark and data scientists workflow
Deeper into Spark
Summary
Chapter 4: Extracting Knowledge through Feature Engineering
The state of the art of feature engineering
Best practices in feature engineering
Feature engineering with Spark
Advanced feature engineering
Summary
Chapter 5: Supervised and Unsupervised Learning by Examples
Machine learning classes
Supervised learning with Spark - an example
Unsupervised learning
Recommender system
Advanced learning and generalizations
Summary
Chapter 6: Building Scalable Machine Learning Pipelines
Spark machine learning pipeline APIs
Cancer-diagnosis pipeline with Spark
Cancer-prognosis pipeline with Spark
Market basket analysis with Spark Core
OCR pipeline with Spark
Topic modeling using Spark MLlib and ML
Credit risk analysis pipeline with Spark
Scaling the ML pipelines
Tips and performance considerations
Summary
Chapter 7: Tuning Machine Learning Models
Details about machine learning model tuning
Typical challenges in model tuning
Evaluating machine learning models
Validation and evaluation techniques
Parameter tuning for machine learning models
Hypothesis testing
Machine learning model selection
Summary
Chapter 8: Adapting Your Machine Learning Models
Adapting machine learning models
The generalization of ML models
Adapting through incremental algorithms
Adapting through reusing ML models
Machine learning in dynamic environments
Summary
Chapter 9: Advanced Machine Learning with Streaming and Graph Data
Developing real-time ML pipelines
Time series and social network analysis
Movie recommendation using Spark
Developing a real-time ML pipeline from streaming
ML pipeline on graph data and semi-supervised graph-based learning
Summary
Chapter 10: Configuring and Working with External Libraries
Third-party ML libraries with Spark
Using external libraries with Spark Core
Time series analysis using the Cloudera Spark-TS package
Configuring SparkR with RStudio
Configuring Hadoop run-time on Windows
Summary

What You Will Learn

  • Get solid theoretical understandings of ML algorithms
  • Configure Spark on cluster and cloud infrastructure to develop applications using Scala, Java, Python, and R
  • Scale up ML applications on large cluster or cloud infrastructures
  • Use Spark ML and MLlib to develop ML pipelines with recommendation system, classification, regression, clustering, sentiment analysis, and dimensionality reduction
  • Handle large texts for developing ML applications with strong focus on feature engineering
  • Use Spark Streaming to develop ML applications for real-time streaming
  • Tune ML models with cross-validation, hyperparameters tuning and train split
  • Enhance ML models to make them adaptable for new data in dynamic and incremental environments

Authors

Table of Contents

Chapter 1: Introduction to Data Analytics with Spark
Spark overview
New computing paradigm with Spark
Spark ecosystem
Spark machine learning libraries
Installing and getting started with Spark
Packaging your application with dependencies
Running a sample machine learning application
References
Summary
Chapter 2: Machine Learning Best Practices
What is machine learning?
Machine learning tasks
Practical machine learning problems
Most widely used machine learning problems
Large scale machine learning APIs in Spark
Practical machine learning best practices
Choosing the right algorithm for your application
Summary
Chapter 3: Understanding the Problem by Understanding the Data
Analyzing and preparing your data
Resilient Distributed Dataset basics
Dataset basics
Dataset from string and typed class
Spark and data scientists workflow
Deeper into Spark
Summary
Chapter 4: Extracting Knowledge through Feature Engineering
The state of the art of feature engineering
Best practices in feature engineering
Feature engineering with Spark
Advanced feature engineering
Summary
Chapter 5: Supervised and Unsupervised Learning by Examples
Machine learning classes
Supervised learning with Spark - an example
Unsupervised learning
Recommender system
Advanced learning and generalizations
Summary
Chapter 6: Building Scalable Machine Learning Pipelines
Spark machine learning pipeline APIs
Cancer-diagnosis pipeline with Spark
Cancer-prognosis pipeline with Spark
Market basket analysis with Spark Core
OCR pipeline with Spark
Topic modeling using Spark MLlib and ML
Credit risk analysis pipeline with Spark
Scaling the ML pipelines
Tips and performance considerations
Summary
Chapter 7: Tuning Machine Learning Models
Details about machine learning model tuning
Typical challenges in model tuning
Evaluating machine learning models
Validation and evaluation techniques
Parameter tuning for machine learning models
Hypothesis testing
Machine learning model selection
Summary
Chapter 8: Adapting Your Machine Learning Models
Adapting machine learning models
The generalization of ML models
Adapting through incremental algorithms
Adapting through reusing ML models
Machine learning in dynamic environments
Summary
Chapter 9: Advanced Machine Learning with Streaming and Graph Data
Developing real-time ML pipelines
Time series and social network analysis
Movie recommendation using Spark
Developing a real-time ML pipeline from streaming
ML pipeline on graph data and semi-supervised graph-based learning
Summary
Chapter 10: Configuring and Working with External Libraries
Third-party ML libraries with Spark
Using external libraries with Spark Core
Time series analysis using the Cloudera Spark-TS package
Configuring SparkR with RStudio
Configuring Hadoop run-time on Windows
Summary

Book Details

ISBN 139781785888748
Paperback476 pages
Read More

Read More Reviews