Exploring and cleaning the data

Now we move on to what might be the most important and time-consuming part of the data science workflow: exploring and cleaning the data. We'll begin by grabbing some basic statistics for the data that we have.

Type the following into another Jupyter notebook cell and run it:

df_raw.describe()

You will see the basic info across all our columns. Note in the following example I grabbed a subset just for practical purposes of displaying it here:

Figure 9.8 – Combined wine basic statistics

There are a few things we can pick out: one is that the mean quality is 5.8, so that is the number that we would want to beat, but if we are looking to be at the higher end of wine quality, we would want to shoot for something above 6, which is the 75th percentile, and nothing gets above a 9, so perhaps that could be our lofty goal.

Note that the quality is a discrete integer in the range 3-9. Should we one-hot encode it...