Learning Couchbase: Design documents and implement real world e-commerce applications with Couchbase

NoSQL is a nonrelational database management system that is different from traditional relational database management systems in significant ways. It is designed for distributed data stores in which there are very large-scale data storage requirements (terabytes and petabytes of data). These types of data storage mechanisms may not require fixed schemas, avoid join operations, and typically scale horizontally.

The main feature of NoSQL is that it is schemaless. There is no fixed schema to store data. Also, there is no join between one or more data records or documents. However, nowadays, most of the NoSQL systems have started providing join features. It allows distributed storage and utilizes computing resources, such as CPU and RAM, spanning across the nodes that are part of the NoSQL cluster.

There are different types of NoSQL data stores. Let's try to cover the four main categories of NoSQL systems in brief:

Electronic data is generated at rapid speed from a variety of sources, such as social media, web server logs, and e-commerce transactions and so on; these include Facebook, Google+, e-commerce websites such as Amazon, eBay, and others. Personal user information, social graphs, geolocation data, user-generated content, and machine logging data are just a few examples of areas in which data has been increasing exponentially. Such data is termed as big data, which usually has a variety of data formats, is generated at a rapid speed, and contains a large set of data. In order to derive information from such big data, large amounts of data have to be processed, for which RDBMS was never designed! The evolution of NoSQL databases is the way to handle such huge data efficiently.

Most of NoSQL databases provide the following benefits:

Since NoSQL is a distributed database system, you need to know a theorem called CAP to understand it better, and take better decisions when the system fails in a distributed environment. Let me explain the CAP theorem to you. There are three important properties of this theorem:

The following is a Venn diagram depicting the CAP theorem:

So, you have understood what the CAP properties signify. The CAP theorem states that in any distributed system it can provide only two features out of these three features. Depending on the type of use cases that the system is intended to address, the database system can choose two out of these three features.

There are a number of database systems available in the IT software market—RDBMS such as MySQL, or NoSQL such as MongoDB, Couchbase, Cassandra, and so on. How do you choose a database system that suits your business requirements? This theorem will help you to decide it. In our context, Couchbase has opted for AP—availability and partition tolerance. So, if your application demands availability and partition tolerance more than consistency, you could opt for Couchbase. However, Couchbase provides a feature called eventual consistency, which will be discussed later in Chapter 6, Retrieving Documents without Keys Using Views. This feature enables the developer to decide the consistency level per operation.

Having understood what NoSQL is all about and why it's a buzzword nowadays, let's try to understand Couchbase, which is the purpose of this book.

Couchbase Server is a persistent, distributed, document-based database that is part of the NoSQL database movement. It combines the capabilities of Apache CouchDB: document-based and indexing-with that of a Membase database, an integrated RAM caching layer, enabling it to support very fast operations, such as create, store, update, and retrieval.

Couchbase Server is a leading NoSQL database project that focuses on distributed database technology and the surrounding ecosystems. It supports both key-value and document-oriented use cases. All components are available under the Apache 2.0 Public License. It can be obtained as packaged software in both an enterprise edition, which is rigorously tested and provides support, and a community edition that do not have support and is open source.

Let's cover some of the main features of Couchbase Server here:

Any operation performed on the Couchbase database system gets stored in the memory, which acts as a caching layer. By default, every document gets stored in the memory for each read, insert, update, and so on, until the memory is full. It's a drop-in replacement for Memcache. However, in order to provide persistency of the record, there is a concept called disk queue. This will flush the record to the disk asynchronously, without impacting the client request. This functionality is provided automatically by the data manager, without any human intervention.

The cluster manager is responsible for node administration and node monitoring within a cluster. Every node within a Couchbase cluster includes the cluster manager component, data storage, and data manager. It manages data storage and retrieval. It contains the memory cache layer, disk persistence mechanism, and query engine.

Couchbase clients use the cluster map provided by the cluster manager to find out which node holds the required data, and then communicate with the data manager on that node to perform database operations.

The Erlang language is used to develop cluster management. Erlang provides a dynamic type system and built-in support for concurrency processes, which are isolated from one another, and are very lightweight in nature. A single Erlang VM, can run a quarter of a million processes. It also provides a lot of modules that help with distributed processing. Moreover, it provides enhancement of debugging and patching live systems is easy compared to any other language.

In RDBMS, we usually encapsulate all relevant data for a particular application in a database namespace. Say, for example, we are developing an e-commerce application. We usually create a database name, e-commerce, which will be used as the logical namespace to store records in a table, such as customer or shopping cart details. It's called a bucket in a Couchbase terminology. So, whenever you want to store any document in a Couchbase cluster, you will be creating a bucket as a logical namespace as a first step. A bucket is an independent virtual container that groups documents logically in a Couchbase cluster, which is equivalent to a database namespace in RDBMS. It can be accessed by various clients in an application. You can also configure features such as security, replication, and so on per bucket. We usually create one database and consolidate all related tables in that namespace in the RDBMS development. Likewise, in Couchbase too, you will usually create one bucket per application and encapsulate all the documents in it.

Views enable indexing and querying by looking inside JSON documents for a key, for ranges of keys, or to aggregate data.

Views are created using incremental MapReduce, which powers indexing. We will discuss this in detail in Chapter 6, Retrieving Documents without Keys Using Views. You can build complex views for your data using the map reduce feature. Views enable us to define materialized views on JSON documents and then query across the dataset.

Using views, you can define primary, simple secondary (the most common use case), complex secondary, tertiary, and composite indexes, as well as aggregations (reduction). It is developed using MapReduce technology. MapReduce functions are written in JavaScript. You will understand more about MapReduce and views in detail in Chapter 6, Retrieving Documents without Keys Using Views.

Cross Data Center Replication (XDCR) is the mechanism provided by Couchbase to replicate documents from one cluster to another. Most of the time, data will be replicated across clusters that are geographically spread out. Usually, when you want to replicate data in a separate data center for disaster recovery or to provide performance by enabling data locality for application, we configure XDCR on a per bucket (per database) basis. When you configure replication across two clusters—say Cluster A is located in Imphal (India) and Cluster B is in Mumbai (India), which are 2,500 kms away from each other—you can specify to replicate only from Cluster A to Cluster B, unidirectional or in both directions, that is, bidirectional. Thus, you are enabling clients to read/write from both the clusters when you enable bidirectional active replication across the clusters. Lastly, you need to remember that this is different from intracluster replication, which occurs within a cluster.

Enough of concepts! Let's install Couchbase so that we can get some hands-on experience. We will install in a Windows environment first. You can download the software from www.couchbase.com.

You can download the enterprise or community edition. We will use the Couchbase 64-bit enterprise edition.

To start the installation wizard, double-click on couchbase-server-enterprise_x86_64_3.0.0.setup.exe. Then, you will see the following window:

Click on Next. The next screen will allow you to select the folder where you want to install Couchbase. In my case, I chose the default location. You can change the directory location if you want; click on the Browse button for that, as shown here:

After selecting the installation folder, you can click on Install, as shown in the following screenshot:

After this, the Couchbase will be installed on your system. You can see the progress of installation in the following screenshot:

It will take some time to install. After a few minutes, you will see the following success screen. Congratulations!!

You need to change some settings before you can use Couchbase. You can access the admin console with http://localhost:8091/index.html. The default port is 8091. You can change this if required.

The Bucket set up

Click on Setup and you will be able to configure Couchbase for your environment:

Server configuration

You can select the location of the databases and indexes with this option. Select the options shown in the preceding screenshot. Since it's the first node we are installing and there is no existing cluster, select Start a new cluster and specify 801 MB RAM, which needs to be allocated to the Couchbase cluster by each node. Then, click on Next:

Sample bucket

Select both the samples provided along with the Couchbase software. After that, click on Next. On the subsequent page, create a default bucket with the following options:

Default bucket

We will explain all these options of bucket settings when we discuss buckets in Chapter 3, Storing Documents in Couchbase Using Buckets. Keep clicking on the Next button until you get the following screen:

Admin credentials

Enter the password, which is root123. It will be used to connect to the Couchbase admin console. Click on Next to finish.

Your Couchbase Server is now running and ready to use!

You can use the rpm command to install Couchbase on Red Hat or CentOS, as follows. Replace the version with the version number of the package downloaded:

# rpm -ivhcouchbase-server-{version}.rpm

For Ubuntu and Debian, use the following:

#dpkg -i couchbase-server{version007D.deb

After the installation is complete, Couchbase starts automatically. You can perform the initial setup by going to http://localhost:8091.

Couchbase gives you the ability to start up and shut down your cluster. Let's take a look at how to achieve this on Linux and Windows.

#/etc/init.d/couchbase-server start
#/etc/init.d/couchbase-server stop

Couchbase Server creates a number of different log files, depending on the component of the system that produced the error, the level and severity of the problem being reported. All these logs are created in a folder. Whenever there is any issue, you can go to the following paths and check the respective logs. Their specific paths are:

Some of the logs that need to be looked into when there is an issue are discussed next.