Examples

Now let's look at a couple of examples in order to understand how to use the new GraphFrames API.

First of all, we need to create a GraphFrames object from data files. Before we do this, we need to install GraphFrames. Fortunately, this is very easy as the package is available on a Maven repository and we can just pass the dependency to the spark-submit and spark-shell commands:

spark-shell --packages graphframes:graphframes:0.5.0-spark2.1-s_2.11

Then, we need to import the required package:

import org.graphframes._

As our graph is based on two CSV files--one for the vertices and one for the edges--we need to create two DataFrames out of them:

val vertex = spark.read.option("header","true").csvgraph1_vertex.csv")
val edges = spark.read.option("header","true").csvgraph1_edges.csv")

Finally, we create a GraphFrames object out of these:

val graph = GraphFrame(vertex, edges)

Example 1 – counting

The graph has been loaded and we know the data volumes in the data files but what about the data content...