Types of RDDs

RDDs can be categorized in multiple categories. Some of the examples include the following:

Hadoop RDD	Shuffled RDD	Pair RDD
Mapped RDD	Union RDD	JSON RDD
Filtered RDD	Double RDD	Vertex RDD

We will not discuss all of them in this chapter, as it is outside the scope of this chapter. But we will discuss one of the important types of RDD: pair RDDs.

Pair RDDs

A pair RDD is a special type of RDD that processes data in the form of key-value pairs. Pair RDD is very useful because it enables basic functionalities such as join and aggregations. Spark provides some special operations on these RDDs in an optimized way. If we recall the examples where we calculated the number of INFO and ERROR messages in...