Basic Terminology and Our End-to-End Example

The philosophy behind a quick-start guide is that the topic at hand is best learned by doing. In this chapter, I will present a quick overview of important vocabulary, concepts, and terminology that you need to get started, and then jump directly into a full end-to-end working example of data mining in Python. Later chapters will flesh out the steps in the working example in more detail.

The following topics will be covered in this chapter:

• Basic data terminology
• Basic statistics
• An end-to-end example of data mining in Python

Basic data terminology

This section is meant to be a quick overview of the terms that you should know before you get started. This list is very streamlined and is not exhaustive. Please refer to the suggested reading in Chapter 1, Data Mining and Getting Started with Python Tools, for wider coverage of domain-specific terminology.

Sample spaces

The sample space is the space that is covered by all the possible outcomes of a measurement. For example, if a feature column in a dataset is populated with the number of days last month that a responder watched television, then the sample space will include all the integers in the {0,1,2...31} set. If a manufacturing tool measures the temperature difference before and after processing...

Basic summary statistics

Practitioners in the field of descriptive analytics use a set of four summary statistics to quickly understand a dataset. With practice, you should be able to strengthen your intuition about each one of these statistical measurements. In fact, it's a great place to start with most problem statements that you will face. The four summary statistics are described as follows:

• Locations: The location or center of the data; this can be measured by the mean (average), median, or mode. The median is the point of delineation in 50% of the data, and the mode is the most occurring points, or largest part of the distribution.
• Spread: How the data is spread around the center; this can be measured with standard deviation, which sums the average distance from the mean of each data point, or variance, which is the square of the deviation.
• Shape: A description...

An end-to-end example of data mining in Python

Let's start with a full end-to-end example demonstrating the topics and strategies covered in the rest of the book. Subsequent chapters will go into further detail on each part of the analytical process. I suggest that you read through this example fully before moving on in the book.

Loading data into memory – viewing and managing with ease using pandas

First, we will need to load data into memory so that Python can interact with it. Pandas will be our data management and manipulation library:

# load data into Pandas
import pandas as pd
df = pd.read_csv("./data/iris.csv")

Let's use some built-in pandas features to do sanity checks on our data load and...

Summary

This chapter covered the basic statistics and data terminology that are required for working in data mining. The final portion of the chapter was dedicated to a full working example, which combined the types of techniques that will be introduced later on in this book. After reading this chapter, you should have a better understanding of the thought processes behind analysis and the common steps taken to address a problem statement that you may encounter in the field. The subsequent chapters will explore each aspect of the example in more depth, with the next chapter focusing on collecting data, loading it into memory, and exploring it with ease.