Preprocessing data

In this chapter, you will learn about preprocessing and that it is an important step when you deal with real-world data. If we want to design a robust visualization that can handle different remote data sources, we need to make sure we process the data beforehand. Datasets can have outliers, can be noisy, can have different representations, can have nested objects, can be flat, and so on. You can see that this list is very long.

Thus, whenever we deal with real data, we need to process it before using it in a visualization; this step is called preprocessing.

Filtering data to remove outliers

As a first preprocessing step, we will look at filters. Filters are array operators that return an array containing a subset of the original elements. In the following figure, we see an example of such an operation:

Filtering a dataset

Let's try an example. We load the following array from a remote data source:

var data = [
  {x: 1, y: 6},
  {x: 2, y: undefined},
  {x: 3, y: 4},
  {x: 4...