Loading and saving data

Loading data into an RDD and Saving an RDD onto an output system both support several different methods. We will cover the most common ones in this section.

Loading data

Loading data into an RDD can be done by using SparkContext. Some of the most common methods are:.

• textFile
• wholeTextFiles
• load from a JDBC datasource

textFile

textFile() can be used to load textFiles into an RDD and each line becomes an element in the RDD.

sc.textFile(name, minPartitions=None, use_unicode=True)

The following is an example of loading a textfile into an RDD using...