Chapter 10. Administration, Tuning, and Monitoring

In any enterprise system, after its implementation, the administrator needs to carry out lot of administrative tasks for the system to perform well. Administration is an ongoing process. From what I observed in my experience, the most daunting task of an administrator was to make systems run smoothly all the time without any hitch. While managing a system administrator team, I usually advise my team to divide administration into three phases:

• Rudimentary phase: In this phase, an administrator has to take care of the basic requirements, such as service availability, disk space, LAN connectivity, and so on. This phase usually consists of monitoring and ensuring that a system is working well and that there are no issues with it. In this phase, there will be a lot of issues, and usually there will be a lot of complaints from the system's users. However, it can be proactively monitored to stabilize the system.

• Advanced phase: In this phase, an...

Overview

The installation and setting up of a Couchbase cluster involves only a few mouse clicks. It's one of the easiest systems to set up that I have ever seen in the 15 years of my career. However, the job of an administrator is not only to install the system but to also monitor it, verify that it is working well, and take pre-emptive action depending on the current capacity and resource consumption. Administration is a vast topic. It can cover networking, operating system, hardware, and so on. We will be only discussing topics related to the Couchbase system in this chapter.

An administrator needs to ensure that proper backup is taken and schedule for critical database systems. Although a Couchbase cluster maintains multiple copies of documents using replication, there is sometimes a need for backup of databases for critical systems, for audit recommendation or compliance, and so on. Thus, we need to ensure that after taking backup of the system, it can be used to restore the system at...

Backup and restoration

One of the regular tasks of an administrator is to take system backups periodically. Whenever there is a disaster, administrators sometimes rely on the backup to restore the data with some data loss, that is, the data that was created after the backup until the disaster or system crash.

Backup is usually of three types:

• Full backup: This backs up the whole database. In this, snapshot of the database, the complete current state is taken. Usually, a full backup is scheduled once in a week.

• Incremental: This is an incremental backup of the data that has changed since the last backup.

• Differential: This is the backup of the transaction that has happened since the last full backup.

cbbackup

Note

Couchbase provides the cbbackup command to take a backup of the Couchbase system. This command enables you to back up an entire cluster, a single node, or a single bucket into a flexible backup structure that allows the restoration of data in the same or different clusters and buckets.

The backup files will be created on disk, and it's recommended to store it in a local disk only. Couchbase takes backups in the SQLite format. This format has some issues while writing to remote file systems. Hence, it's not recommended to direct the output of cbbackup to a remote file system.

Backing up all nodes and all buckets

The following command will initiate a full backup (the -m full option) of all the buckets in the cluster. The backup will be dumped in the D:/mybackup1509 folder:

cbbackup -m full http://10.72.47.31:8091 D:/mybackup1509

cbbackup – full backup

The preceding diagram displays the progress status when the backup is initiated.

If you want to take a differential backup, change the...

Rebalancing

The process of distributing documents across the nodes in the cluster is called rebalancing.

As an administrators, we need to expand the cluster size by adding nodes to a cluster whenever there are resource constraints due to increases in the application load. Sometimes, we need to trim the resources of the cluster, for example, removing nodes for upgrades of software, hardware maintenance, and so on. Whenever there is a change in cluster resources, the administrator needs to perform balancing of data across the nodes. This is a deliberate process that needs to be done manually by the administrator. In Couchbase, data is distributed across nodes using the vBucket structure. During the rebalancing process, the documents stored in the vBuckets are redistributed among the nodes available at the time of the process to reflect the actual resources of the cluster.

This is an online activity; that is, the Couchbase cluster will be running and serving the clients requests during rebalancing...

Adding and removing a node from the cluster

Your project manager calls you up and asks you to add one node to the cluster to increase the capacity of the cluster. So, what would you do? You need to install the Couchbase server on the new server, and in the installation wizard, you need to ensure that you select the Join a cluster now option, as shown in the following screenshot. Then, you need to enter the IP address of the existing cluster (in our case, this is 10.72.47.31) and the login credentials to connect to the cluster:

Joining to the cluster –the configuration

Then click on Next. You will be shown the following screen:

Cluster overview

In the preceding screenshot, the Couchbase Server Message sections display that a new node is associated with the cluster and requires rebalancing to distribute data across nodes.

Click on the Server Nodes tab and then on Pending Rebalance, as shown in this screenshot:

Server nodes

In the admin console, you can see the node that we just added to the cluster...

Performing a bulk set

Often, an administrator needs to upload data to the database from a different system or, at the time of database migration, from one system to another system. Couchbase provides the cbdocloader tool to upload a group of JSON documents. These can be a directory or a ZIP file. The following command uploads JSON documents contained in books.zip to LCBucket:

cbdocloader -n localhost:8091 -u Administrator -p root123 -b LCBucket books.zip

Monitoring

Couchbase provides multiple ways to monitor a cluster through the admin web console, the CLI, and the REST API.

You can verify the underlying server processes using the cbcollect_info utility. This tool is executed per node and collects information about the overall health of the cluster. The output of the command will be in a .zip file. It contains numerous log files. These log files will provide comprehensive information about the cluster:

cbcollect_info collect_info.zip

The collect info result

Monitoring startup

Server startup, also known as warmup is a process in which the server loads data from the disk to the RAM before it can handle clients' requests. During this process, Couchbase won't be serving any request from the clients. After the warmup process completes, the data is available for clients to read and write. This can take some time, depending on the size of the data. You can monitor the warmup process using the following command:

cbstats localhost:11210 warmup

The cbstats...

Best practices

Let's understand some best practices and considerations that you, as an administrator, need to keep in mind while configuring a Couchbase cluster.

Cluster design

While designing a Couchbase cluster, we need to think in terms of RAM, CPU cores, the number of nodes, and disk I/O:

• RAM: The most crucial component in a Couchbase cluster is the RAM. The optimal response will be provided by the cluster whenever most of the working set data required by applications is located in the RAM. Hence, proper sizing of the RAM is required for good performance. The administrator should calculate the proper working set data and allocate enough memory for storing it to provide optimal performance, since all reads and writes will occur directly through the memory.

• Nodes and cores: How many nodes should be there in a cluster? It's better to have more nodes than fewer nodes with high-capacity hardware resources. After calculating the required memory, you can find out the number of nodes required...

Summary

In this chapter, we discussed the various options available for the administration, tuning, and monitoring of a Couchbase cluster. You understood how to use backup and restoration tools. Subsequently, we added a node to the existing cluster and performed rebalancing of data across nodes. In the last section, we discussed some best practices and design considerations to keep in mind while designing a cluster.

WOW! Finally, we have covered most of the features provided by the Couchbase cluster to successfully host a database in a production environment. You understood the architecture, installed it, and configured it. You also learned how to connect to the cluster using the Java API.

We will not be discussing anything new in the next chapter. In it, we will develop an e-commerce application using Java APIs, incorporating all the features you have learned in the last 10 chapters.