Reader small image

You're reading from  HBase Administration Cookbook

Product typeBook
Published inAug 2012
PublisherPackt
ISBN-139781849517140
Edition1st Edition
Right arrow
Author (1)
Yifeng Jiang
Yifeng Jiang
author image
Yifeng Jiang

Yifeng Jiang is a Hadoop and HBase Administrator and Developer at Rakutenthe largest e-commerce company in Japan. After graduating from the University of Science and Technology of China with a B.S. in Information Management Systems, he started his career as a professional software engineer, focusing on Java development. In 2008, he started looking over the Hadoop project. In 2009, he led the development of his previous company's display advertisement data infrastructure using Hadoop and Hive. In 2010, he joined his current employer, where he designed and implemented the Hadoop- and HBase-based, large-scale item ranking system. He is also one of the members of the Hadoop team in the company, which operates several Hadoop/HBase clusters
Read more about Yifeng Jiang

Right arrow

Cluster replication


HBase supports cluster replication, which is a way to copy data between the HBase clusters. For example, it can be used as a way to easily ship edits from a real-time frontend cluster to a batch purpose cluster on the backend.

The basic architecture of an HBase replication is very practical. The master cluster captures write ahead log (WAL), and puts replicable Key/Values (edits of the column family with replication support) from the log into the replication queue. The replication message is then sent to the peer cluster, and then replayed on that cluster using its normal HBase client API. The master cluster also keeps the current position of the WAL being replicated in ZooKeeper for failure recovery.

Because the HBase replication is done asynchronously, the clusters participating in the replication can be geographically distant. It is not a problem if the connections between them are offline for some time, as the master cluster will track the replication, and recover...

lock icon
The rest of the page is locked
Previous PageNext Chapter
You have been reading a chapter from
HBase Administration Cookbook
Published in: Aug 2012Publisher: PacktISBN-13: 9781849517140

Author (1)

author image
Yifeng Jiang

Yifeng Jiang is a Hadoop and HBase Administrator and Developer at Rakutenthe largest e-commerce company in Japan. After graduating from the University of Science and Technology of China with a B.S. in Information Management Systems, he started his career as a professional software engineer, focusing on Java development. In 2008, he started looking over the Hadoop project. In 2009, he led the development of his previous company's display advertisement data infrastructure using Hadoop and Hive. In 2010, he joined his current employer, where he designed and implemented the Hadoop- and HBase-based, large-scale item ranking system. He is also one of the members of the Hadoop team in the company, which operates several Hadoop/HBase clusters
Read more about Yifeng Jiang