Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Fast Data Processing with Spark 2 - Third Edition

You're reading from  Fast Data Processing with Spark 2 - Third Edition

Product type Book
Published in Oct 2016
Publisher Packt
ISBN-13 9781785889271
Pages 274 pages
Edition 3rd Edition
Languages
Author (1):
Holden Karau Holden Karau
Profile icon Holden Karau

Table of Contents (18) Chapters

Fast Data Processing with Spark 2 Third Edition
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Installing Spark and Setting Up Your Cluster Using the Spark Shell Building and Running a Spark Application Creating a SparkSession Object Loading and Saving Data in Spark Manipulating Your RDD Spark 2.0 Concepts Spark SQL Foundations of Datasets/DataFrames – The Proverbial Workhorse for DataScientists Spark with Big Data Machine Learning with Spark ML Pipelines GraphX

The first example - graph


As our first example, let's create a simple graph. Remember the Giraffe Graph we talked about? Let's create one that is shown in the following figure:

This is a graph from the lecture given by Prof. Jeffrey D. Ullman at Stanford University on graphs and social Networks (http://web.stanford.edu/class/cs246/handouts.html). It is a giraffe graph with two strong cliques connected by a weak edge. The numbers are the betweenness centrality of each node, as calculated by the G-N algorithm.

Note

Fun facts about betweenness centrality: It shows how many paths an edge is part of; that is, its relevancy. High betweenness centrality is the sign of a bottleneck, a point of single failure; such edges need HA and probably alternate paths for rerouting, and they are susceptible to parasite infections and good candidates for a cut!

From the GraphX computational model, you can see that betweenness centrality would be represented as an edge property.

The nodes A, B,... are people, and...

lock icon The rest of the chapter is locked
arrow left Previous Chapter
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime}