Instant Parallel Processing with Gearman [Instant]

This title is available as an eBook only
Instant Parallel Processing with Gearman [Instant]
eBook: $19.99
Formats: PDF, PacktLib, ePub and Mobi formats
save 15%!
Print & eBook also available on:
Learn in an Instant - Short, Fast, Focused
Table of Contents
Sample Chapters
  • Learn something new in an Instant! A short, fast, focused guide delivering immediate results
  • Build a cluster of managers, workers, and clients using Gearman to scale your application
  • Understand how to reduce single-points-of-failure in your distributed applications
  • Build clients and workers to process data in the background and provide real-time updates to your frontend

Book Details

Language : English
eBook : 58 pages
Release Date : July 2013
ISBN : 1783284072
ISBN 13 : 9781783284078
Author(s) : John Ewart
Topics and Technologies : All Books, Application Development, Instant, Open Source

Table of Contents

Instant Parallel Processing with Gearman
  • Instant Parallel Processing with Gearman
    • So, what is Gearman?
      • Distinguishing features of Gearman
      • Overview of components
      • The conversation between the actors
        • Usecase – image processing
    • Quick Start – building your first components
      • Step 1 – running a server
      • Step 2 – downloading a client library
        • Your first client
        • Writing the client
      • Step 3 – our first worker
      • Step 4 – varying priorities
      • Step 5 – background tasks
      • And that's it
    • Top 5 features you need to know about
      • Job handling
        • Completed versus successful
        • Unique identifiers and coalescing
        • The client
        • The worker
        • Making it work
      • Scaling your system
        • Running multiple managers
        • Writing a client that supports multiple managers
        • Submitting jobs to multiple managers
        • Handling a manager being terminated
        • Processing jobs from multiple managers
      • MapReduce
        • The shared library
        • A simple program
        • The client
        • The worker
        • Parallelizing the pipeline
        • Scaling this solution
        • Processing a large data file
      • Providing job updates
        • Data updates
        • Status updates
        • Building a processing pipeline
        • Using multiple languages
      • Persistence engines
        • Why is this important?
        • Persistent versus non-persistent
        • How safe is safe?
    • People and places you should get to know
      • Client libraries
      • Community

John Ewart

John Ewart is a system architect, software developer, and lecturer. He has designed and taught courses at a variety of institutions including the University of California, California State University, and local community colleges covering a wide range of computer science topics including Java, data structures and algorithms, operating systems fundamentals, Unix and Linux systems administration, and web application development. In addition to working and teaching, he maintains and contributes to a number of open source projects. He currently resides in Redmond, Washington, with his wife, Mary, and their two children.
Sorry, we don't have any reviews for this title yet.

Submit Errata

Please let us know if you have found any errors not listed on this list by completing our errata submission form. Our editors will check them and add them to this list. Thank you.

Sorry, there are currently no downloads available for this title.

Frequently bought together

Instant Parallel Processing with Gearman [Instant] +    Gradle Effective Implementation Guide =
50% Off
the second eBook
Price for both: £20.45

Buy both these recommended eBooks together and get 50% off the cheapest eBook.

What you will learn from this book

  • Develop clients and workers to process data
  • Use Gearman to implement a map-reduce cluster
  • Take existing libraries and use them with Gearman
  • Learn how Gearman stores jobs and available persistence options
  • Build a cluster of managers, workers, and clients to scale your application
  • Discover ways to use Gearman to avoid single-point-of-failure in your system

In Detail

Many of today’s applications need to be able to process large volumes of data, and vertical scaling has its limits both in terms of prohibitive cost and hardware limits. Gearman is an open source job manager that is well-suited to building horizontally scalable systems, from map-reduce algorithms to simple data processors capable of handling massive amounts of information.

Instant Gearman is a practical, hands-on guide to getting started with building an open source job management server system that is built to grow. Learn the basics of building a distributed application that spans multiple components and learn how Gearman fits into building an application that scales from one to hundreds of components that interact to process data. With Gearman, you can build software that scales horizontally as your need for computation increases.

Instant Gearman has in-depth examples and a step-by-step approach to building distributed systems, helping you to build systems that are scalable and modular in their approach to processing data.

Once you are comfortable with building simple workers and clients, learn how to build a cluster of managers and see how to reduce single-point-of-failure in your architecture. Next, build a simple map-reduce application using Gearman and scale it up from a single instance to multiple parallel processing components.


Get to grips with a new technology, understand what it is and what it can do for you, and then get to work with the most important features and tasks. This is a quick tutorial allowing you to quickly increase the efficiency of your system by intelligently reducing its workload where possible and learn techniques for designing your solutions to take advantage of Gearman’s architecture.

The technology covered in this book is usable by almost any modern programming language and any type of application, be it web, desktop, or mobile applications.

Who this book is for

This book targets developers who are interested in adding distributed data processing to their applications. The examples in this book are written using Ruby, but are documented thoroughly to allow developers comfortable with other languages to understand how to use them in a language of their choice. Intermediate programmers with no previous experience in distributed computing will also be able to easily follow the content.

Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software