You're reading from Hands-On Data Preprocessing in Python

Product type Book

Published in Jan 2022

Publisher Packt

ISBN-13 9781801072137

Pages 602 pages

Edition 1st Edition

Languages

Concepts

Big Data

Author (1):

Roy Jafari

Table of Contents (24) Chapters

Preface

1. Part 1:Technical Needs

2. Chapter 1: Review of the Core Modules of NumPy and Pandas

3. Chapter 2: Review of Another Core Module – Matplotlib

4. Chapter 3: Data – What Is It Really?

5. Chapter 4: Databases

6. Part 2: Analytic Goals

7. Chapter 5: Data Visualization

8. Chapter 6: Prediction

9. Chapter 7: Classification

10. Chapter 8: Clustering Analysis

11. Part 3: The Preprocessing

12. Chapter 9: Data Cleaning Level I – Cleaning Up the Table

13. Chapter 10: Data Cleaning Level II – Unpacking, Restructuring, and Reformulating the Table

14. Chapter 11: Data Cleaning Level III – Missing Values, Outliers, and Errors

15. Chapter 12: Data Fusion and Data Integration

16. Chapter 13: Data Reduction

17. Chapter 14: Data Transformation and Massaging

18. Part 4: Case Studies

19. Chapter 15: Case Study 1 – Mental Health in Tech

20. Chapter 16: Case Study 2 – Predicting COVID-19 Hospitalizations

21. Chapter 17: Case Study 3: United States Counties Clustering Analysis

22. Chapter 18: Summary, Practice Case Studies, and Conclusions

23. Other Books You May Enjoy

The whys of data transformation and massaging

Data transformation comes at the very last stage of data preprocessing, right before using the analytic tools. At this stage of data preprocessing, the dataset already has the following characteristics.

Data cleaning: The dataset is cleaned at all three cleaning levels (Chapters 9–11).
Data integration: All the potentially beneficial data sources are recognized and a dataset that includes the necessary information is created (Chapter 12, Data Fusion and Integration).
Data reduction: If needed, the size of the dataset has been reduced (Chapter 13, Data Reduction).

At this stage of data preprocessing, we may have to make some changes to the data before moving to the analyzing stage. The dataset will undergo the changes for one of the following reasons: we will call them necessity, correctness, and effectiveness. The following list provides more detail for each reason.