HBase Administration Cookbook

Table of Contents

• Preface

• Chapter 1: Setting Up HBase Cluster
• Introduction
• Quick start
• Getting ready on Amazon EC2
• Setting up Hadoop
• Setting up ZooKeeper
• Changing the kernel settings
• Setting up HBase
• Basic Hadoop/ZooKeeper/HBase configurations
• Setting up multiple High Availability (HA) masters

• Chapter 2: Data Migration
• Introduction
• Importing data from MySQL via single client
• Importing data from TSV files using the bulk load tool
• Writing your own MapReduce job to import data
• Precreating regions before moving data into HBase

• Chapter 3: Using Administration Tools
• Introduction
• HBase Master web UI
• Using HBase Shell to manage tables
• Using HBase Shell to access data in HBase
• Using HBase Shell to manage the cluster
• Executing Java methods from HBase Shell
• Row counter
• WAL tool—manually splitting and dumping WALs
• HFile tool—viewing textualized HFile content
• HBase hbck—checking the consistency of an HBase cluster
• Hive on HBase—querying HBase using a SQL-like language

• Chapter 4: Backing Up and Restoring HBase Data
• Introduction
• Full shutdown backup using distcp
• Using CopyTable to copy data from one table to another
• Exporting an HBase table to dump files on HDFS
• Restoring HBase data by importing dump files from HDFS
• Backing up NameNode metadata
• Backing up region starting keys
• Cluster replication

• Chapter 5: Monitoring and Diagnosis
• Introduction
• Showing the disk utilization of HBase tables
• Setting up Ganglia to monitor an HBase cluster
• OpenTSDB—using HBase to monitor an HBase cluster
• Setting up Nagios to monitor HBase processes
• Using Nagios to check Hadoop/HBase logs
• Simple scripts to report the status of the cluster
• Hot region—write diagnosis

• Chapter 6: Maintenance and Security
• Introduction
• Enabling HBase RPC DEBUG-level logging
• Graceful node decommissioning
• Adding nodes to the cluster
• Rolling restart
• Simple script for managing HBase processes
• Simple script for making deployment easier
• Kerberos authentication for Hadoop and HBase
• Configuring HDFS security with Kerberos
• HBase security configuration

• Chapter 7: Troubleshooting
• Introduction
• Troubleshooting tools
• Handling the XceiverCount error
• Handling the "too many open files" error
• Handling the "unable to create new native thread" error
• Handling the "HBase ignores HDFS client configuration" issue
• Handling the ZooKeeper client connection error
• Handling the ZooKeeper session expired error
• Handling the HBase startup error on EC2

• Chapter 8: Basic Performance Tuning
• Introduction
• Setting up Hadoop to spread disk I/O
• Using network topology script to make Hadoop rack-aware
• Mounting disks with noatime and nodiratime
• Setting vm.swappiness to 0 to avoid swap
• Java GC and HBase heap settings
• Using compression
• Managing compactions
• Managing a region split

• Chapter 9: Advanced Configurations and Tuning
• Introduction
• Benchmarking HBase cluster with YCSB
• Increasing region server handler count
• Precreating regions using your own algorithm
• Avoiding update blocking on write-heavy clusters
• Tuning memory size for MemStores
• Client-side tuning for low latency systems
• Configuring block cache for column families
• Increasing block cache size on read-heavy clusters
• Client side scanner setting
• Tuning block size to improve seek performance
• Enabling Bloom Filter to improve the overall throughput

• Index