Search icon CANCEL
Subscription
0
Cart icon
Cart
Close icon
You have no products in your basket yet
Save more on your purchases!
Savings automatically calculated. No voucher code required
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
The Supervised Learning Workshop - Second Edition
The Supervised Learning Workshop - Second Edition

The Supervised Learning Workshop: Predict outcomes from data by building your own powerful predictive models with machine learning in Python, Second Edition

By Blaine Bateman , Ashish Ranjan Jha , Benjamin Johnston , Ishita Mathur
$26.99 $17.99
Book Feb 2020 532 pages 2nd Edition
eBook
$26.99 $17.99
Print
$38.99
Subscription
$15.99 Monthly
eBook
$26.99 $17.99
Print
$38.99
Subscription
$15.99 Monthly

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Buy Now
Table of content icon View table of contents Preview book icon Preview Book

The Supervised Learning Workshop - Second Edition

2. Exploratory Data Analysis and Visualization

Overview

This chapter takes us through how to perform exploration and analysis on a new dataset. By the end of this chapter, you will be able to explain the importance of data exploration and communicate the summary statistics of a dataset. You will visualize patterns in missing values in data and be able to replace null values appropriately. You will be equipped to identify continuous features, categorical features and visualize distributions of values across individual variables. You will also be able to describe and analyze relationships between different types of variables using correlation and visualizations.

Introduction

Say we have a problem statement that involves predicting whether a particular earthquake caused a tsunami. How do we decide what model to use? What do we know about the data we have? Nothing! But if we don't know and understand our data, chances are we'll end up building a model that's not very interpretable or reliable. When it comes to data science, it's important to have a thorough understanding of the data we're dealing with, in order to generate features that are highly informative and, consequently, to build accurate and powerful models. To acquire this understanding, we perform an exploratory analysis of the data to see what the data can tell us about the relationships between the features and the target variable (the value that you are trying to predict using the other variables). Getting to know our data will even help us interpret the model we build and identify ways we can improve its accuracy. The approach we take to achieve this is...

Exploratory Data Analysis (EDA)

Exploratory data analysis (EDA) is defined as a method to analyze datasets and sum up their main characteristics to derive useful conclusions, often with visual methods.

The purpose of EDA is to:

  • Discover patterns within a dataset
  • Spot anomalies
  • Form hypotheses regarding the behavior of data
  • Validate assumptions

Everything from basic summary statistics to complex visualizations helps us gain an intuitive understanding of the data itself, which is highly important when it comes to forming new hypotheses about the data and uncovering what parameters affect the target variable. Often, discovering how the target variable varies across a single feature gives us an indication of how important a feature might be, and a variation across a combination of several features helps us to come up with ideas for new informative features to engineer.

Most explorations and visualizations are intended to understand the relationship between...

Summary Statistics and Central Values

In order to find out what our data really looks like, we use a technique known as data profiling. This is defined as the process of examining the data available from an existing information source (for example, a database or a file) and collecting statistics or informative summaries about that data. The goal is to make sure that you understand your data well and are able to identify any challenges that the data may pose early on in the project, which is done by summarizing the dataset and assessing its structure, content, and quality.

Data profiling includes collecting descriptive statistics and data types. Common data profile commands include those you have seen previously, including data.describe(), data.head(), and data.tail(). You can also use data.info(), which tells you how many non-null values there are in each column, along with the data type of the values (non-numeric types are represented as object types).

Exercise 2.01: Summarizing...

Missing Values

When there is no value (that is, a null value) recorded for a particular feature in a data point, we say that the data is missing. Having missing values in a real dataset is inevitable; no dataset is ever perfect. However, it is important to understand why the data is missing, and whether there is a factor that has affected the loss of data. Appreciating and recognizing this allows us to handle the remaining data in an appropriate manner. For example, if the data is missing randomly, then it's highly likely that the remaining data is still representative of the population. However, if the missing data is not random in nature and we assume that it is, it could bias our analysis and subsequent modeling.

Let's look at the common reasons (or mechanisms) for missing data:

  • Missing Completely at Random (MCAR): Values in a dataset are said to be MCAR if there is no correlation whatsoever between the value missing and any other recorded variable or external...

Distribution of Values

In this section, we'll look at how individual variables behave—what kind of values they take, what the distribution across those values is, and how those distributions can be represented visually.

Target Variable

The target variable can either have values that are continuous (in the case of a regression problem) or discrete (as in the case of a classification problem). The problem statement we're looking at in this chapter involves predicting whether an earthquake caused a tsunami, that is, the flag_tsunami variable, which takes on two discrete values only—making it a classification problem.

One way of visualizing how many earthquakes resulted in tsunamis and how many didn't involves the use of a bar chart, where each bar represents a single discrete value of the variable, and the height of the bars is equal to the count of the data points having the corresponding discrete value. This gives us a good comparison of the absolute...

Relationships within the Data

There are two reasons why it is important to find relationships between variables in the data:

  • Establishing which features are potentially important can be deemed essential, since finding ones that have a strong relationship with the target variable will aid in the feature selection process.
  • Finding relationships between different features themselves can be useful since variables in the dataset are usually never completely independent of every other variable and this can affect our modeling in a number of ways.

Now, there are a number of ways in which we can visualize these relationships, and this really depends on the types of variable we are trying to find the relationship between, and how many we are considering as part of the equation or comparison.

Relationship between Two Continuous Variables

Establishing a relationship between two continuous variables is basically seeing how one varies as the value of the other is increased...

Summary

In this chapter, we started by talking about why data exploration is an important part of the modeling process and how it can help in not only preprocessing the dataset for the modeling process but also help us engineer informative features and improve model accuracy. This chapter focused on not only gaining a basic overview of the dataset and its features but also gaining insights by creating visualizations that combine several features. We looked at how to find the summary statistics of a dataset using core functionality from pandas. We looked at how to find missing values and talked about why they're important while learning how to use the Missingno library to analyze them and the pandas and scikit-learn libraries to impute the missing values. Then, we looked at how to study the univariate distributions of variables in the dataset and visualize them for both categorical and continuous variables using bar charts, pie charts, and histograms. Lastly, we learned how to...

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Explore the fundamentals of supervised machine learning and its applications
  • Learn how to label and process data correctly using Python libraries
  • Gain a comprehensive overview of different machine learning algorithms used for building prediction models

Description

Would you like to understand how and why machine learning techniques and data analytics are spearheading enterprises globally? From analyzing bioinformatics to predicting climate change, machine learning plays an increasingly pivotal role in our society. Although the real-world applications may seem complex, this book simplifies supervised learning for beginners with a step-by-step interactive approach. Working with real-time datasets, you’ll learn how supervised learning, when used with Python, can produce efficient predictive models. Starting with the fundamentals of supervised learning, you’ll quickly move to understand how to automate manual tasks and the process of assessing date using Jupyter and Python libraries like pandas. Next, you’ll use data exploration and visualization techniques to develop powerful supervised learning models, before understanding how to distinguish variables and represent their relationships using scatter plots, heatmaps, and box plots. After using regression and classification models on real-time datasets to predict future outcomes, you’ll grasp advanced ensemble techniques such as boosting and random forests. Finally, you’ll learn the importance of model evaluation in supervised learning and study metrics to evaluate regression and classification tasks. By the end of this book, you’ll have the skills you need to work on your real-life supervised learning Python projects.

What you will learn

Import NumPy and pandas libraries to assess the data in a Jupyter Notebook Discover patterns within a dataset using exploratory data analysis Using pandas to find the summary statistics of a dataset Improve the performance of a model with linear regression analysis Increase the predictive accuracy with decision trees such as k-nearest neighbor (KNN) models Plot precision-recall and ROC curves to evaluate model performance

Product Details

Country selected

Publication date : Feb 28, 2020
Length 532 pages
Edition : 2nd Edition
Language : English
ISBN-13 : 9781800209046
Category :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Buy Now

Product Details


Publication date : Feb 28, 2020
Length 532 pages
Edition : 2nd Edition
Language : English
ISBN-13 : 9781800209046
Category :

Table of Contents

9 Chapters
Preface Chevron down icon Chevron up icon
1. Fundamentals Chevron down icon Chevron up icon
2. Exploratory Data Analysis and Visualization Chevron down icon Chevron up icon
3. Linear Regression Chevron down icon Chevron up icon
4. Autoregression Chevron down icon Chevron up icon
5. Classification Techniques Chevron down icon Chevron up icon
6. Ensemble Modeling Chevron down icon Chevron up icon
7. Model Evaluation Chevron down icon Chevron up icon
Appendix Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Empty star icon Empty star icon Empty star icon Empty star icon Empty star icon 0
(0 Ratings)
5 star 0%
4 star 0%
3 star 0%
2 star 0%
1 star 0%
Top Reviews
No reviews found
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.