Search icon
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Hadoop Real-World Solutions Cookbook - Second Edition

You're reading from  Hadoop Real-World Solutions Cookbook - Second Edition

Product type Book
Published in Mar 2016
Publisher
ISBN-13 9781784395506
Pages 290 pages
Edition 2nd Edition
Languages
Author (1):
Tanmay Deshpande Tanmay Deshpande
Profile icon Tanmay Deshpande

Table of Contents (18) Chapters

Hadoop Real-World Solutions Cookbook Second Edition
Credits
About the Author
Acknowledgements
About the Reviewer
www.PacktPub.com
Preface
Getting Started with Hadoop 2.X Exploring HDFS Mastering Map Reduce Programs Data Analysis Using Hive, Pig, and Hbase Advanced Data Analysis Using Hive Data Import/Export Using Sqoop and Flume Automation of Hadoop Tasks Using Oozie Machine Learning and Predictive Analytics Using Mahout and R Integration with Apache Spark Hadoop Use Cases Index

Clustering text data using K-Means


In this recipe, we are going to take a look at how to use Mahout to cluster text data using Mahout's implementation of the K-Means algorithm. K-Means is very popular clustering algorithm; you can read more about it at https://en.wikipedia.org/wiki/K-means_clustering.

Getting ready

To perform this recipe, you should have a running Hadoop cluster as well as the latest version of Mahout installed on it.

How to do it...

In this recipe, we are going to use Mahout's K Means algorithm to cluster the text data that is available. To do this, we first need to get some text data and copy it to HDFS:

hadoop fs –mkdir /kmeans
hadoop fs –put mydata.txt /kmeans/input

In order to execute the K-Means job on the given data, we first need to convert it into sequential files and from these sequential files to TF-IDF vectors. Mahout provides built-in utilities to perform these actions. The following are the commands to do this.

To convert text data into a sequential file, here is...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime}