Clean Data

Save time by discovering effortless strategies for cleaning, organizing, and manipulating your data

Clean Data

This ebook is included in a Mapt subscription
Megan Squire

2 customer reviews
Save time by discovering effortless strategies for cleaning, organizing, and manipulating your data
$0.00
$18.00
$44.99
$29.99p/m after trial
RRP $35.99
RRP $44.99
Subscription
eBook
Print + eBook
Start 30 Day Trial
Subscribe and access every Packt eBook & Video.
 
  • 4,000+ eBooks & Videos
  • 40+ New titles a month
  • 1 Free eBook/Video to keep every month
Start Free Trial
 
Code Files
Preview in Mapt

Book Details

ISBN 139781785284014
Paperback272 pages

Book Description

Is much of your time spent doing tedious tasks such as cleaning dirty data, accounting for lost data, and preparing data to be used by others? If so, then having the right tools makes a critical difference, and will be a great investment as you grow your data science expertise.

The book starts by highlighting the importance of data cleaning in data science, and will show you how to reap rewards from reforming your cleaning process. Next, you will cement your knowledge of the basic concepts that the rest of the book relies on: file formats, data types, and character encodings. You will also learn how to extract and clean data stored in RDBMS, web files, and PDF documents, through practical examples.

At the end of the book, you will be given a chance to tackle a couple of real-world projects.

Table of Contents

Chapter 1: Why Do You Need Clean Data?
A fresh perspective
The data science process
Communicating about data cleaning
Our data cleaning environment
An introductory example
Summary
Chapter 2: Fundamentals – Formats, Types, and Encodings
File formats
Archiving and compression
Data types, nulls, and encodings
Summary
Chapter 3: Workhorses of Clean Data – Spreadsheets and Text Editors
Spreadsheet data cleaning
Text editor data cleaning
An example project
Summary
Chapter 4: Speaking the Lingua Franca – Data Conversions
Quick tool-based conversions
Converting with PHP
Converting with Python
The example project
Summary
Chapter 5: Collecting and Cleaning Data from the Web
Understanding the HTML page structure
Method one – Python and regular expressions
Method two – Python and BeautifulSoup
Method three – Chrome Scraper
Example project – Extracting data from e-mail and web forums
Summary
Chapter 6: Cleaning Data in PDF Files
Why is cleaning PDF files difficult?
Try simple solutions first – copying
Another technique to try – pdfMiner
Third choice – Tabula
When all else fails – the fourth technique
Summary
Chapter 7: RDBMS Cleaning Techniques
Getting ready
Step one – download and examine Sentiment140
Step two – clean for database import
Step three – import the data into MySQL in a single table
Step four – clean the & character
Step five – clean other mystery characters
Step seven – separate user mentions, hashtags, and URLs
Step eight – cleaning for lookup tables
Summary
Chapter 8: Best Practices for Sharing Your Clean Data
Preparing a clean data package
Documenting your data
Setting terms and licenses for your data
Publicizing your data
Summary
Chapter 9: Stack Overflow Project
Step one – posing a question about Stack Overflow
Step two – collecting and storing the Stack Overflow data
Step three – cleaning the data
Step four – analyzing the data
Step five – visualizing the data
Step six – problem resolution
Moving from test tables to full tables
Summary
Chapter 10: Twitter Project
Step one – posing a question about an archive of tweets
Step two – collecting the data
Step three – data cleaning
Step four – simple data analysis
Step five – visualizing the data
Step six – problem resolution
Moving this process into full (non-test) tables
Summary

What You Will Learn

  • Understand the role of data cleaning in the overall data science process
  • Learn the basics of file formats, data types, and character encodings to clean data properly
  • Master critical features of the spreadsheet and text editor for organizing and manipulating data
  • Convert data from one common format to another, including JSON, CSV, and some special-purpose formats
  • Implement three different strategies for parsing and cleaning data found in HTML files on the Web
  • Reveal the mysteries of PDF documents and learn how to pull out just the data you want
  • Develop a range of solutions for detecting and cleaning bad data stored in an RDBMS
  • Create your own clean data sets that can be packaged, licensed, and shared with others
  • Use the tools from this book to complete two real-world projects using data from Twitter and Stack Overflow

Authors

Table of Contents

Chapter 1: Why Do You Need Clean Data?
A fresh perspective
The data science process
Communicating about data cleaning
Our data cleaning environment
An introductory example
Summary
Chapter 2: Fundamentals – Formats, Types, and Encodings
File formats
Archiving and compression
Data types, nulls, and encodings
Summary
Chapter 3: Workhorses of Clean Data – Spreadsheets and Text Editors
Spreadsheet data cleaning
Text editor data cleaning
An example project
Summary
Chapter 4: Speaking the Lingua Franca – Data Conversions
Quick tool-based conversions
Converting with PHP
Converting with Python
The example project
Summary
Chapter 5: Collecting and Cleaning Data from the Web
Understanding the HTML page structure
Method one – Python and regular expressions
Method two – Python and BeautifulSoup
Method three – Chrome Scraper
Example project – Extracting data from e-mail and web forums
Summary
Chapter 6: Cleaning Data in PDF Files
Why is cleaning PDF files difficult?
Try simple solutions first – copying
Another technique to try – pdfMiner
Third choice – Tabula
When all else fails – the fourth technique
Summary
Chapter 7: RDBMS Cleaning Techniques
Getting ready
Step one – download and examine Sentiment140
Step two – clean for database import
Step three – import the data into MySQL in a single table
Step four – clean the & character
Step five – clean other mystery characters
Step seven – separate user mentions, hashtags, and URLs
Step eight – cleaning for lookup tables
Summary
Chapter 8: Best Practices for Sharing Your Clean Data
Preparing a clean data package
Documenting your data
Setting terms and licenses for your data
Publicizing your data
Summary
Chapter 9: Stack Overflow Project
Step one – posing a question about Stack Overflow
Step two – collecting and storing the Stack Overflow data
Step three – cleaning the data
Step four – analyzing the data
Step five – visualizing the data
Step six – problem resolution
Moving from test tables to full tables
Summary
Chapter 10: Twitter Project
Step one – posing a question about an archive of tweets
Step two – collecting the data
Step three – data cleaning
Step four – simple data analysis
Step five – visualizing the data
Step six – problem resolution
Moving this process into full (non-test) tables
Summary

Book Details

ISBN 139781785284014
Paperback272 pages
Read More
From 2 reviews

Read More Reviews