What do you get with a Packt Subscription?

Free for first 7 days. $15.99 p/m after that. Cancel any time!

Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!

50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

Thousands of reference materials covering every tech concept you need to stay up to date.

Subscribe now

View plans & pricing

MongoDB High Availability

Chapter 1. Understanding the MongoDB Architecture and Processes

To be able to diagnose a MongoDB server or change the default preferences to provide better performance in a database, we need to understand the primitive MongoDB settings and management tools. MongoDB consists of binary files and services that make the infrastructure of the server, and each file performs a specific task.

In this chapter, we will go through the MongoDB processes to discover why they exist and what each process does exactly.

Utilizing MongoDB components

This section contains a brief description of the MongoDB components, file names, and the main purpose of each of them. We will further discuss each item in detail. The following diagram shows you the four main components of MongoDB:

MongoDB components can mainly be classified into the following categories:

Core components
Import and export tools
Diagnostic tools
File storage (GridFS) tools

Each file can be placed into one of the aforementioned categories. The core component files are used to run the MongoDB server and start it. The files are also used to manage the MongoDB server from the command-line interface or manage clustering tasks.

Using import and export tools, developers can create dump files from their database in different formats such as BSON, JSON or CSV and restore them into another database again. Certain tools are used to create or restore BSON files, and some tools are responsible for generating and importing other common formats such as JSON or CSV.

MongoDB has many built-in diagnostic tools to manage and control the currently running server. Finally, we can use GridFS tools to interact with filesystem and GridFS components. In the next section, we will give you more details for each category and the processes inside them.

Understanding the core components

Inside this category, you can find the core processes that are required to start the MongoDB server. The MongoDB engine uses these tools to accept requests from different clients.

The components and executable files in this group are as follows:

mongod
mongo
mongos

Understanding mongod

The mongod component is the primary process that is needed for MongoDB to start the server. It manages requests or queries and handles connections.

The following screenshot is the result of executing mongod from the command-line interface:

The preceding screenshot illustrates the result of running mongod from the command line, that is, you finally have a MongoDB server running on port 27017 and a web interface on port 28017, which is the default port.

Like other MongoDB commands, in order to see the parameters of the mongod process, you can simply run following command:

mongod --help

Here, you can see the result of running the mongod command with the --help parameter. This is shown in the following screenshot:

Note

To read more about the command and figure out what each parameter does, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongod/.

Utilizing mongo

After starting the database server, we need to interact with it to issue a command, run commands, run queries, or get reports. In the MongoDB structure, the mongo file is responsible for this task. This is an interactive JavaScript shell that you can utilize from your command-line environment.

Using this executable file, database administrators and also developers can manage the database server or get available databases and collections.

The following screenshot is the result of running the mongo command:

Fortunately, the mongo component provides internal help so that developers can use it to get more information for each command.

The following screenshot shows you the result of running the help command:

Note

To acquire more information about MongoDB and all its parameters, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongo/.

Learning about mongos

The mongos instance is responsible for routing read/write commands to shards in a sharded cluster. In fact, all processes and applications will connect to this instance and run queries, then the mongos instance will route commands to available shards. The applications will not interact with shards directly. To run the mongos instance, we can provide sharded cluster configurations using a config file or command-line parameters.

In further chapters, we will discuss this process in detail.

Note

To gain more knowledge about the mongos command and its parameters, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongos/.

Import and export tools

The tools in this group are used to export or restore files from the database. Mainly, components in this category are classified into two different groups. This includes components that work with binary data or interact with common data formats such as JSON, CSV, and so on.

Using import and export tools

In order to import or export binary data from and into the database, developers can use tools in this group. We use tools and utilities in this category to create or restore backups for our MongoDB server. In further chapters, we will discuss the backup strategies in detail.

The following are the tools that are available to perform these kinds of tasks:

mongodump
mongorestore
bsondump
mongooplog

Understanding mongodump

This process comes in handy when developers or system administrators want to create a dump file from a database in the binary format. This utility and other related tools are useful for MongoDB backup strategies.

The mongodump process is able to retrieve data from either mongod or mongos processes.

Note

For more information on mongodump, visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongodump/.

Utilizing mongorestore

This process is used to restore and write the binary files that are generated by the mongodump process into the database server. To restore data, mongorestore can establish a new database or use an existing database. Just like the mongodump instance, mongorestore can connect to a mongos instance or it can connect directly to the mongod process.

Note

To read more about mongostore, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongorestore/.

Learning about bsondump

The bsondump process is used to convert the BSON format data to common data formats such as JSON. Essentially, bsondump comes in handy when developers want to convert dump files that are generated by mongodump to human-readable formats.

A very simple usage of this command is shown in the following command line:

bsondump data.bson>data.json

Understanding mongooplog

The mongooplog is a utility that duplicates oplog from one server to another server, for instance, in order to perform a migration task. In order to perform the migration operation, mongooplog accepts two parameters, the from and to server addresses.

Note

What is oplog?

The oplog or operation log is a capped collection (a fixed-sized collection) of data that keeps the record of all data altering operations. In further chapters, we will explain this feature in detail.

The following is a simple usage of this command:

mongooplog --from server1 --host server2

The preceding command line will connect to the MongoDB instance of server1 and copy the entire oplog to server2 instance.

Using data tools

In this group, we have two utilities that help us generate or import data in human-readable formats such as JSON or CSV into or from the MongoDB instance. These are mentioned in the following lists:

mongoimport
mongoexport

Understanding mongoexport

In order to export data in JSON or CSV formats from the MongoDB instance, developers can use this utility.

The following is a simple usage of this command:

mongoexport --db mydb --collection posts --out export.json

The preceding command line will connect to a local instance of MongoDB, retrieve all records from the posts collection of the mydb database in the JSON format, and write all outputs to the export.json file.

Utilizing mongoimport

The mongoimport utility can help you import the produced export files in JSON, CSV or TSV formats into the MongoDB instance. The export files can be generated from either mongoexport or from other third-party export tools.

The following example is a basic usage of this command:

mongoimport --db mydb --collection posts --file export.json

The preceding command line will import the export.json entries into the posts collection of the mydb database. The same instruction can be used for other data formats using the --type option.

Diagnostic tools

One of the important tools of a database system is diagnostic tools. Fortunately, MongoDB has built-in diagnostic tools that enable developers to diagnose the server or get a brief report from the system.

We have the following utilities placed in this group:

mongostat
mongotop
mongosniff
mongoperf

In the next sections, you can read a brief description of each utility.

Learning about mongostat

This tool produces a brief summary of relevant statistics of the currently running MongoDB instances, either the mongod or mongos instance.

The following screenshot illustrates the output of this tool:

The preceding screenshot shows you the number of queries, update, insert, and delete operations from the database every second.

The following bullet list gives you a brief description for each column:

insert: This refers to the number of insert operations per second.
query: This refers to the number of queries per second.
update: This refers to the number of update operations per second.
delete: This refers to the number of delete operations per second.
getmore: This refers to the number of getmore operations (that is, the it command in mongo shell) per second.
command: This refers to the number of executed commands since the last mongostat call.
flushes: This refers to the number of fsync operations at the time of the last mongostat execution. The fsync operation is a system call that flushes all dirty in-memory pages to the disk.
mapped: This refers to the total amount of data mapped in megabytes.
vsize: This refers to the amount of virtual memory in megabytes used by the process at the time of the last mongostat execution.
res: This refers to the amount of resident memory in megabytes used by the process at the time of the last mongostat execution.
locked: This refers to the percentage of time in a global write lock.
idx miss: This refers to the percentage of index access attempts that required a page fault.
qr: This refers to the number of clients in the queue that are waiting for read operations.
qw: This refers to the number of clients in the queue that are waiting for write operations.
ar: This refers to the number of clients that execute read operations.
aw: This refers to the number of clients that execute write operations.
netIn: This refers to the traffic received by the MongoDB instance in bytes.
netOut: This refers to the traffic sent by the MongoDB instance in bytes.
conn: This refers to the current total option connections.

The refresh interval can be changed using the following command:

mongostat [options] [sleep time]

Utilizing mongotop

The mongotop utility provides you with a mechanism to get information about time spent on read/write operations. This command is similar to Unix's top command.

The following screenshot shows you a simple usage of mongotop:

Understanding mongosniff

The mongosniff is a tool that is used to fetch live MongoDB collection statistics. While inserting or querying data from the MongoDB instances, you can run the mongosniff command and connect it to your MongoDB instance to see what the database does.

Note

Please note that in order to use this utility, you should install the libpcap library first. To install the libpcap library, please visit its official website at http://www.tcpdump.org/#documentation.

A simple usage of the mongosniff tool is as follows:

sudo mongosniff  --source NET lo0

The preceding command line will listen to the loopback interface (localhost). This interface is lo0 in Mac OS systems and lo for other operating systems, usually. You can get the list for your network interfaces using the ifconfig command. If you're using Windows as the operating system, you can get the list of network interfaces using the following command:

ipconfig /all

Utilizing mongoperf

The mongoperf tool represents the disk I/O performance. It checks the I/O in a specified interval and illustrates it. This utility can be used independent of MongoDB.

File storage (GridFS) tools

With the help of GridFS, MongoDB can be used as a filesystem. The processes in this section are used to manage and control the GridFS feature.

There is one process in this category, which is as follows:

mongofiles

Understanding mongofiles

This utility enables developers to retrieve files that are stored in the database in the GridFS collection. The mongofiles utility come in handy when developers need to interact with files stored in the database from the command-line environment.

The usage of this command looks like the following:

mongofiles <options> <commands> <filename>

The following example is a simple usage of this utility:

mongofiles -d mydb list

The preceding command line will retrieve all files in the GridFS collection from the mydb database.

Note

For more information on mongofiles, please visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongofiles/.

What do you get with a Packt Subscription?

Free for first 7 days. $15.99 p/m after that. Cancel any time!

Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!

50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

Thousands of reference materials covering every tech concept you need to stay up to date.

Subscribe now

View plans & pricing

MongoDB High Availability: Design and implement a highly available server using the latest features of MongoDB.

What do you get with a Packt Subscription?

Product Details

Key benefits

What you will learn

What do you get with a Packt Subscription?

Product Details

Packt Subscriptions

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

Authors (1)

FAQs