On a write-heavy HBase cluster, you may observe an unstable write speed. Most of the writes are very fast, while some are slow. For an online system, this unstable write speed is not acceptable even when average speed is very fast.
This situation is probably caused by the following two reasons:
Split/compaction makes the cluster very high load
Updates are blocked by region server
As we described in Chapter 8, Basic Performance Tuning you can avoid the split/compaction issues by disabling the automatic split/compaction and invoking them at low load time.
Grep your region server logs, if you find many messages saying "Blocking updates for ...", it is possible that many updates were blocked, and those updates might have poor response time.
To fix this issue, we need to tune both server side and client side configurations to gain a stable write speed. We will describe the most important server side tuning to avoid update blocking in this recipe.
