Apache Kafka 1.0 Cookbook

Simplify real-time data processing by leveraging the power of Apache Kafka 1.0
Preview in Mapt

Apache Kafka 1.0 Cookbook

Raúl Estrada

Simplify real-time data processing by leveraging the power of Apache Kafka 1.0

Quick links: > What will you learn?> Table of content

eBook
$19.60
RRP $27.99
Save 29%
Print + eBook
$34.99
RRP $34.99
What do I get with a Mapt Pro subscription?
  • Unlimited access to all Packt’s 5,000+ eBooks and Videos
  • Early Access content, Progress Tracking, and Assessments
  • 1 Free eBook or Video to download and keep every month after trial
What do I get with an eBook?
  • Download this book in EPUB, PDF, MOBI formats
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
What do I get with Print & eBook?
  • Get a paperback copy of the book delivered to you
  • Download this book in EPUB, PDF, MOBI formats
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
What do I get with a Video?
  • Download this Video course in MP4 format
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
$19.60
$34.99
RRP $27.99
RRP $34.99
eBook
Print + eBook

Frequently bought together


Apache Kafka 1.0 Cookbook Book Cover
Apache Kafka 1.0 Cookbook
$ 27.99
$ 19.60
DevOps with Kubernetes Book Cover
DevOps with Kubernetes
$ 39.99
$ 28.00
Buy 2 for $35.00
Save $32.98
Add to Cart

Book Details

ISBN 139781787286849
Paperback250 pages

Book Description

Apache Kafka provides a unified, high-throughput, low-latency platform to handle real-time data feeds. This book will show you how to use Kafka efficiently, and contains practical solutions to the common problems that developers and administrators usually face while working with it.

This practical guide contains easy-to-follow recipes to help you set up, configure, and use Apache Kafka in the best possible manner. You will use Apache Kafka Consumers and Producers to build effective real-time streaming applications. The book covers the recently released Kafka version 1.0, the Confluent Platform and Kafka Streams. The programming aspect covered in the book will teach you how to perform important tasks such as message validation, enrichment and composition.Recipes focusing on optimizing the performance of your Kafka cluster, and integrate Kafka with a variety of third-party tools such as Apache Hadoop, Apache Spark, and Elasticsearch will help ease your day to day collaboration with Kafka greatly. Finally, we cover tasks related to monitoring and securing your Apache Kafka cluster using tools such as Ganglia and Graphite.

If you're looking to become the go-to person in your organization when it comes to working with Apache Kafka, this book is the only resource you need to have.

Table of Contents

Chapter 1: Configuring Kafka
Introduction
Installing Kafka
Running Kafka
Configuring Kafka brokers
Configuring Kafka topics
Creating a message console producer
Creating a message console consumer
Configuring the broker settings
Configuring threads and performance
Configuring the log settings
Configuring the replica settings
Configuring the ZooKeeper settings
Configuring other miscellaneous parameters
Chapter 2: Kafka Clusters
Introduction
Configuring a single-node single-broker cluster – SNSB
SNSB – creating a topic, producer, and consumer
Configuring a single-node multiple-broker cluster – SNMB
SNMB – creating a topic, producer, and consumer
Configuring a multiple-node multiple-broker cluster – MNMB
Chapter 3: Message Validation
Introduction
Modeling the events
Setting up the project
Reading from Kafka
Writing to Kafka
Running ProcessingApp
Coding the validator
Running the validator
Chapter 4: Message Enrichment
Introduction
Geolocation extractor
Geolocation enricher
Currency price extractor
Currency price enricher
Running the currency price enricher
Modeling the events
Setting up the project
Open weather extractor
Location temperature enricher
Running the location temperature enricher
Chapter 5: The Confluent Platform
Introduction
Installing the Confluent Platform
Using Kafka operations
Monitoring with the Confluent Control Center
Using the Schema Registry
Using the Kafka REST Proxy
Using Kafka Connect
Chapter 6: Kafka Streams
Introduction
Setting up the project
Running the streaming application
Chapter 7: Managing Kafka
Introduction
Managing consumer groups
Dumping log segments
Importing ZooKeeper offsets
Using the GetOffsetShell
Using the JMX tool
Using the MirrorMaker tool
Replaying log producer
Using state change log merger
Chapter 8: Operating Kafka
Introduction
Adding or removing topics
Modifying message topics
Implementing a graceful shutdown
Balancing leadership
Expanding clusters
Increasing the replication factor
Decommissioning brokers
Checking the consumer position
Chapter 9: Monitoring and Security
Introduction
Monitoring server statistics
Monitoring producer statistics
Monitoring consumer statistics
Connecting with the help of Graphite
Monitoring with the help of Ganglia
Implementing authentication using SSL
Implementing authentication using SASL/Kerberos
Chapter 10: Third-Party Tool Integration
Introduction
Moving data between Kafka nodes with Flume
Writing to an HDFS cluster with Gobblin
Moving data from Kafka to Elastic with Logstash
Connecting Spark streams and Kafka
Ingesting data from Kafka to Storm
Pushing data from Kafka to Elastic
Inserting data from Kafka to SolrCloud
Building a Kafka producer with Akka
Building a Kafka consumer with Akka
Storing data in Cassandra
Running Kafka on Mesos
Reading Kafka with Apache Beam
Writing to Kafka from Apache Beam

What You Will Learn

  • Install and configure Apache Kafka 1.0 to get optimal performance
  • Create and configure Kafka Producers and Consumers
  • Operate your Kafka clusters efficiently by implementing the mirroring technique
  • Work with the new Confluent platform and Kafka streams, and achieve high availability with Kafka
  • Monitor Kafka using tools such as Graphite and Ganglia
  • Integrate Kafka with third-party tools such as Elasticsearch, Logstash, Apache Hadoop, Apache Spark, and more

Authors

Table of Contents

Chapter 1: Configuring Kafka
Introduction
Installing Kafka
Running Kafka
Configuring Kafka brokers
Configuring Kafka topics
Creating a message console producer
Creating a message console consumer
Configuring the broker settings
Configuring threads and performance
Configuring the log settings
Configuring the replica settings
Configuring the ZooKeeper settings
Configuring other miscellaneous parameters
Chapter 2: Kafka Clusters
Introduction
Configuring a single-node single-broker cluster – SNSB
SNSB – creating a topic, producer, and consumer
Configuring a single-node multiple-broker cluster – SNMB
SNMB – creating a topic, producer, and consumer
Configuring a multiple-node multiple-broker cluster – MNMB
Chapter 3: Message Validation
Introduction
Modeling the events
Setting up the project
Reading from Kafka
Writing to Kafka
Running ProcessingApp
Coding the validator
Running the validator
Chapter 4: Message Enrichment
Introduction
Geolocation extractor
Geolocation enricher
Currency price extractor
Currency price enricher
Running the currency price enricher
Modeling the events
Setting up the project
Open weather extractor
Location temperature enricher
Running the location temperature enricher
Chapter 5: The Confluent Platform
Introduction
Installing the Confluent Platform
Using Kafka operations
Monitoring with the Confluent Control Center
Using the Schema Registry
Using the Kafka REST Proxy
Using Kafka Connect
Chapter 6: Kafka Streams
Introduction
Setting up the project
Running the streaming application
Chapter 7: Managing Kafka
Introduction
Managing consumer groups
Dumping log segments
Importing ZooKeeper offsets
Using the GetOffsetShell
Using the JMX tool
Using the MirrorMaker tool
Replaying log producer
Using state change log merger
Chapter 8: Operating Kafka
Introduction
Adding or removing topics
Modifying message topics
Implementing a graceful shutdown
Balancing leadership
Expanding clusters
Increasing the replication factor
Decommissioning brokers
Checking the consumer position
Chapter 9: Monitoring and Security
Introduction
Monitoring server statistics
Monitoring producer statistics
Monitoring consumer statistics
Connecting with the help of Graphite
Monitoring with the help of Ganglia
Implementing authentication using SSL
Implementing authentication using SASL/Kerberos
Chapter 10: Third-Party Tool Integration
Introduction
Moving data between Kafka nodes with Flume
Writing to an HDFS cluster with Gobblin
Moving data from Kafka to Elastic with Logstash
Connecting Spark streams and Kafka
Ingesting data from Kafka to Storm
Pushing data from Kafka to Elastic
Inserting data from Kafka to SolrCloud
Building a Kafka producer with Akka
Building a Kafka consumer with Akka
Storing data in Cassandra
Running Kafka on Mesos
Reading Kafka with Apache Beam
Writing to Kafka from Apache Beam

Book Details

ISBN 139781787286849
Paperback250 pages
Read More

Read More Reviews

Recommended for You

DevOps with Kubernetes Book Cover
DevOps with Kubernetes
$ 39.99
$ 28.00
Architecting Modern Java EE Applications Book Cover
Architecting Modern Java EE Applications
$ 39.99
$ 28.00
Statistics for Machine Learning Book Cover
Statistics for Machine Learning
$ 39.99
$ 28.00
TensorFlow 1.x Deep Learning Cookbook Book Cover
TensorFlow 1.x Deep Learning Cookbook
$ 35.99
$ 25.20
Cloud Computing with Apache CloudStack: Run your own cloud [Video] Book Cover
Cloud Computing with Apache CloudStack: Run your own cloud [Video]
$ 193.99
$ 164.90
Salesforce CRM Admin Cookbook - Second Edition Book Cover
Salesforce CRM Admin Cookbook - Second Edition
$ 39.99
$ 28.00