Chapter 3. Parallel Computing – MPI on the Raspberry Pi

In this chapter we will be investigating the technology known as MPICH. MPICH is an implementation of the Message Passing Interface standard which we briefly touched upon in Chapter 1, Clusters, Parallel Computing, and Raspberry Pi – A Brief Background.

So what subject area do we cover in relation to this technology?

First we will compare MPICH to an alternative implementation of MPI called OpenMPI. Following this we will install and then set up MPICH on our Raspberry Pi (RPi) and run a test application to check if it is working. After this we will clone our SD card and set up our second Raspberry Pi. This gives us the opportunity to execute a test application on two Raspberry Pis and see a calculation of Π being run in parallel.

Finally we will write some simple applications to demonstrate how MPI works.

MPI – Message Passing Interface

As we explained in Chapter 1, Clusters, Parallel Computing, and Raspberry Pi – A Brief Background, the Message Passing Interface is a language-independent message-passing communication protocol designed for parallel computing applications.

The standard's beginning can be found in the early 1990's when a number of academics and figures from industry combined their efforts to design a message passing system that would aid parallel computing application development.

The MPI standard defines a core set of routines that can be used by a programmer in order to distribute their application and handle passing back the results of the executed code seamlessly. In MPI's early days, C and Fortran were the languages most closely associated with it; however, Java and Python among others have also gone on to offer support. We will now touch upon two of the C and Fortran implementations.

MPI implementations – MPICH and OpenMPI

There are two prominent implementations of MPI that can be used on the Raspberry Pi. These are: OpenMPI and MPICH.

OpenMPI is an open source implementation of MPI maintained by a collection of industry and academic partners. It has been implemented on a number of the world's top 500 supercomputers including the Japanese K computer.

OpenMPI's origins can be found in several other projects including the University of Tennessee's FT-MPI project, Indiana University's LAM/MPI, University of Stuttgart's PCX-MPI, and LA-MPI from Los Alamos National Laboratory in the USA.

You can find out more about the technology at the official website:

http://www.open-mpi.org/

MPICH, originally standing for Message Passing Interface CHameleon, is an implementation of the MPI standard that supports C, C++, and Fortran applications. It was initially developed in the early 90's to provide feedback on implementation issues to the MPI forum.

The MPICH Wiki providing more background...

Creating an environment and downloading MPICH

Before we install any software, we are going to create a number of directories under our account. These will be used for installing MPICH:

If you are not connected to your Master Raspberry Pi, log back in.

From inside your home directory you can then create the following folder:

mkdir mpich3

Navigate into mpich3 and create the following two directories:

mkdir build install

The mpich3 is the directory where we will be installing the MPICH software as well. Feel free to change the numeric value in the directory name to match the major version number of the MPICH software you are downloading.

Our next task is to grab the latest package from the MPICH downloads link:http://www.mpich.org/downloads/

You can use wget to perform this task, make sure the version number is the latest:

wget http://www.mpich.org/static/downloads/3.0.4/mpich-3.0.4.tar.gz

Once the tar.gz file has been downloaded we can unzip it into the mpich3 directory:

tar xvfz mpich-3.0.4.tar.gz

You...

Writing an MPI-based application

The following application is a simple Hello World style program that will demonstrate some of the features of MPI.

A list of supported MPI-based functions and datatypes can be reviewed at:

http://www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf

You will need to log into the Master Raspberry Pi to start with.

On this machine create a new file in the code directory under mpich3 called hello_rpi.c. For example:

vim /home/pi/mpich3/code/hello_rpi.c

To this file we are now going to add the following code:

/*
Hello RPI implemented using MPI
*/

#include <stdio.h>
#include <mpi.h>

Here we include the MPI-specific header directive: mpi.h, which will give us access to the MPI library.

Following this we can add the main function to the code. Copy and paste the following under the header directives:

int main(int argc, char *argv[])
{
  int rpi; // The raspberry pi node
  int totalrpi; // The total number of rpi's

  MPI_Init(&argc, &argv);
  MPI_Comm_rank...

Summary

In this chapter, we explored the subject of MPI using the C programming language to give us a taste of parallel computing. We also wrote our first two parallel applications and set up a second Raspberry Pi node by cloning the Master Raspberry Pi's SD card.

In comparison to MPI we can look at a Java-based technology called Hadoop. In the next chapter we shall start this task.