Apache Spark Graph Processing

More Information
Learn
  • Write, build and deploy Spark applications with the Scala Build Tool.
  • Build and analyze large-scale network datasets
  • Analyze and transform graphs using RDD and graph-specific operations
  • Implement new custom graph operations tailored to specific needs.
  • Develop iterative and efficient graph algorithms using message aggregation and Pregel abstraction
  • Extract subgraphs and use it to discover common clusters
  • Analyze graph data and solve various data science problems using real-world datasets.
About

Apache Spark is the next standard of open-source cluster-computing engine for processing big data. Many practical computing problems concern large graphs, like the Web graph and various social networks. The scale of these graphs - in some cases billions of vertices, trillions of edges - poses challenges to their efficient processing. Apache Spark GraphX API combines the advantages of both data-parallel and graph-parallel systems by efficiently expressing graph computation within the Spark data-parallel framework.

This book will teach the user to do graphical programming in Apache Spark, apart from an explanation of the entire process of graphical data analysis. You will journey through the creation of graphs, its uses, its exploration and analysis and finally will also cover the conversion of graph elements into graph structures.

This book begins with an introduction of the Spark system, its libraries and the Scala Build Tool. Using a hands-on approach, this book will quickly teach you how to install and leverage Spark interactively on the command line and in a standalone Scala program. Then, it presents all the methods for building Spark graphs using illustrative network datasets. Next, it will walk you through the process of exploring, visualizing and analyzing different network characteristics. This book will also teach you how to transform raw datasets into a usable form. In addition, you will learn powerful operations that can be used to transform graph elements and graph structures. Furthermore, this book also teaches how to create custom graph operations that are tailored for specific needs with efficiency in mind. The later chapters of this book cover more advanced topics such as clustering graphs, implementing graph-parallel iterative algorithms and learning methods from graph data.

Features
  • Find solutions for every stage of data processing from loading and transforming graph data to
  • Improve the scalability of your graphs with a variety of real-world applications with complete Scala code.
  • A concise guide to processing large-scale networks with Apache Spark.
Page Count 148
Course Length 4 hours 26 minutes
ISBN 9781784391805
Date Of Publication 9 Sep 2015

Authors

Rindra Ramamonjison

Rindra Ramamonjison is a fourth year PhD student of electrical engineering at the University of British Columbia, Vancouver. He received his master's degree from Tokyo Institute of Technology. He has played various roles in many engineering companies, within telecom and finance industries. His primary research interests are machine learning, optimization, graph processing, and statistical signal processing. Rindra is also the co-organizer of the Vancouver Spark Meetup.