Preface

Although there are many resources available on the Web for Hadoop, most stop at the surface or provide a solution for a specific problem. Instant MapReduce Patterns – Hadoop Essentials How-to is a concise introduction to Hadoop and programming with MapReduce. It is aimed to get you started and give an overall feel to programming with Hadoop so that you will have a solid foundation to dig deep into each type of MapReduce problem, as needed.

What this book covers

Writing a word count application using Java (Simple) describes how to write a word count program using Java, without MapReduce. We will use this to compare and contrast against the MapReduce model.

Writing a word count application with MapReduce and running it (Simple) explains how to write the word count using MapReduce and how to run it using the Hadoop local mode.

Installing Hadoop in a distributed setup and running a word count application (Simple) describes how to install Hadoop in a distributed setup and run the above Wordcount job in a distributed setup.

Writing a formatter (Intermediate) explains how to write a Hadoop data formatter to read the Amazon data format as a record instead of reading data line by line.

Analytics – drawing a frequency distribution with MapReduce (Intermediate) describes how to process Amazon data with MapReduce, generate data for a histogram, and plot it using gnuplot.

Relational operations – join two datasets with MapReduce (Advanced) describes how to join two datasets using MapReduce.

Set operations with MapReduce (Intermediate) describes how to process Amazon data and perform the set difference with MapReduce. Further, it will discuss how other set operations can also be implemented using similar methods.

Cross correlation with MapReduce (Intermediate) explains how to use MapReduce to count the number of times two items occur together (cross correlation).

Simple search with MapReduce (Intermediate) describes how to process Amazon data and implement a simple search using an inverted index.

Simple graph operations with MapReduce (Advanced) describes how to perform a graph traversal using MapReduce.

Kmeans with MapReduce (Advanced) describes how to cluster a dataset using the Kmeans algorithm. Clustering groups the data into several groups such that items in each group are similar and items in different groups are different according to some distance measure.

What you need for this book

To try out this book, you need access to a Linux or Mac computer with JDK 1.6 installed.

Who this book is for

For big data enthusiasts and would-be Hadoop programmers. The book for Java programmers who either have not worked with Hadoop at all, or who know Hadoop and MapReduce bit, but are not sure how to deepen their understanding.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text are shown as follows: " Verify the installation by listing processes through the ps | grep java command."

A block of code is set as follows:

public void map(Object key, Text value, Context context) {
    StringTokenizeritr = new   
        StringTokenizer(value.toString());
    while (itr.hasMoreTokens()) {
       word.set(itr.nextToken());
       context.write(word, one);
    }
}

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
job.setInputFormatClass(ItemSalesDataFormat.class);
FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);

Any command-line input or output is written as follows:

>bin/hadoopdfs -mkdir /data/
>bin/hadoopdfs -mkdir /data/amazon-dataset
>bin/hadoopdfs -put <SAMPLE_DIR>/amazon-meta.txt /data/amazon-dataset/
>bin/hadoopdfs -ls /data/amazon-dataset

New terms and important words are shown in bold.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <[e-mail]>, and mention the book title via the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.

Piracy

Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <[e-mail]>with a link to the suspected pirated material.

We appreciate your help in protecting our authors, and our ability to bring you valuable content.

Questions

You can contact us at <[e-mail]> if you are having a problem with any aspect of the book, and we will do our best to address it.