To be able to diagnose a MongoDB server or change the default preferences to provide better performance in a database, we need to understand the primitive MongoDB settings and management tools. MongoDB consists of binary files and services that make the infrastructure of the server, and each file performs a specific task.
In this chapter, we will go through the MongoDB processes to discover why they exist and what each process does exactly.
This section contains a brief description of the MongoDB components, file names, and the main purpose of each of them. We will further discuss each item in detail. The following diagram shows you the four main components of MongoDB:
MongoDB components can mainly be classified into the following categories:
Import and export tools
File storage (GridFS) tools
Each file can be placed into one of the aforementioned categories. The core component files are used to run the MongoDB server and start it. The files are also used to manage the MongoDB server from the command-line interface or manage clustering tasks.
Using import and export tools, developers can create dump files from their database in different formats such as BSON, JSON or CSV and restore them into another database again. Certain tools are used to create or restore BSON files, and some tools are responsible for generating and importing other common formats such as JSON or CSV.
MongoDB has many built-in diagnostic tools to manage and control the currently running server. Finally, we can use GridFS tools to interact with filesystem and GridFS components. In the next section, we will give you more details for each category and the processes inside them.
Inside this category, you can find the core processes that are required to start the MongoDB server. The MongoDB engine uses these tools to accept requests from different clients.
The components and executable files in this group are as follows:
mongod component is the primary process that is needed for MongoDB to start the server. It manages requests or queries and handles connections.
The following screenshot is the result of executing
mongod from the command-line interface:
The preceding screenshot illustrates the result of running
mongod from the command line, that is, you finally have a MongoDB server running on port 27017 and a web interface on port 28017, which is the default port.
Like other MongoDB commands, in order to see the parameters of the
mongod process, you can simply run following command:
Here, you can see the result of running the
mongod command with the
--help parameter. This is shown in the following screenshot:
To read more about the command and figure out what each parameter does, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongod/.
After starting the database server, we need to interact with it to issue a command, run commands, run queries, or get reports. In the MongoDB structure, the
Using this executable file, database administrators and also developers can manage the database server or get available databases and collections.
The following screenshot is the result of running the
mongo component provides internal help so that developers can use it to get more information for each command.
The following screenshot shows you the result of running the
To acquire more information about MongoDB and all its parameters, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongo/.
mongos instance is responsible for routing read/write commands to shards in a sharded cluster. In fact, all processes and applications will connect to this instance and run queries, then the
mongos instance will route commands to available shards. The applications will not interact with shards directly. To run the
mongos instance, we can provide sharded cluster configurations using a
config file or command-line parameters.
In further chapters, we will discuss this process in detail.
To gain more knowledge about the
mongos command and its parameters, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongos/.
The tools in this group are used to export or restore files from the database. Mainly, components in this category are classified into two different groups. This includes components that work with binary data or interact with common data formats such as JSON, CSV, and so on.
In order to import or export binary data from and into the database, developers can use tools in this group. We use tools and utilities in this category to create or restore backups for our MongoDB server. In further chapters, we will discuss the backup strategies in detail.
The following are the tools that are available to perform these kinds of tasks:
This process comes in handy when developers or system administrators want to create a dump file from a database in the binary format. This utility and other related tools are useful for MongoDB backup strategies.
mongodump process is able to retrieve data from either
For more information on mongodump, visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongodump/.
This process is used to restore and write the binary files that are generated by the
mongodump process into the database server. To restore data,
mongorestore can establish a new database or use an existing database. Just like the
mongorestore can connect to a
mongos instance or it can connect directly to the
To read more about
mongostore, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongorestore/.
bsondump process is used to convert the BSON format data to common data formats such as JSON. Essentially,
bsondump comes in handy when developers want to convert dump files that are generated by
mongodump to human-readable formats.
A very simple usage of this command is shown in the following command line:
mongooplog is a utility that duplicates
oplog from one server to another server, for instance, in order to perform a migration task. In order to perform the migration operation,
mongooplog accepts two parameters, the
to server addresses.
What is oplog?
oplog or operation log is a capped collection (a fixed-sized collection) of data that keeps the record of all data altering operations. In further chapters, we will explain this feature in detail.
The following is a simple usage of this command:
mongooplog --from server1 --host server2
The preceding command line will connect to the MongoDB instance of
server1 and copy the entire
In this group, we have two utilities that help us generate or import data in human-readable formats such as JSON or CSV into or from the MongoDB instance. These are mentioned in the following lists:
In order to export data in JSON or CSV formats from the MongoDB instance, developers can use this utility.
The following is a simple usage of this command:
mongoexport --db mydb --collection posts --out export.json
The preceding command line will connect to a local instance of MongoDB, retrieve all records from the
posts collection of the
mydb database in the JSON format, and write all outputs to the
mongoimport utility can help you import the produced export files in JSON, CSV or TSV formats into the MongoDB instance. The export files can be generated from either
mongoexport or from other third-party export tools.
The following example is a basic usage of this command:
mongoimport --db mydb --collection posts --file export.json
The preceding command line will import the
export.json entries into the
posts collection of the
mydb database. The same instruction can be used for other data formats using the
One of the important tools of a database system is diagnostic tools. Fortunately, MongoDB has built-in diagnostic tools that enable developers to diagnose the server or get a brief report from the system.
We have the following utilities placed in this group:
In the next sections, you can read a brief description of each utility.
This tool produces a brief summary of relevant statistics of the currently running MongoDB instances, either the
The following screenshot illustrates the output of this tool:
The preceding screenshot shows you the number of queries, update, insert, and delete operations from the database every second.
The following bullet list gives you a brief description for each column:
insert: This refers to the number of insert operations per second.
update: This refers to the number of update operations per second.
delete: This refers to the number of delete operations per second.
getmore: This refers to the number of
getmoreoperations (that is, the
itcommand in mongo shell) per second.
command: This refers to the number of executed commands since the last
flushes: This refers to the number of
fsyncoperations at the time of the last
fsyncoperation is a system call that flushes all dirty in-memory pages to the disk.
mapped: This refers to the total amount of data mapped in megabytes.
vsize: This refers to the amount of virtual memory in megabytes used by the process at the time of the last
res: This refers to the amount of resident memory in megabytes used by the process at the time of the last
locked: This refers to the percentage of time in a global write lock.
idx miss: This refers to the percentage of index access attempts that required a page fault.
qr: This refers to the number of clients in the queue that are waiting for read operations.
qw: This refers to the number of clients in the queue that are waiting for write operations.
ar: This refers to the number of clients that execute read operations.
aw: This refers to the number of clients that execute write operations.
netIn: This refers to the traffic received by the MongoDB instance in bytes.
netOut: This refers to the traffic sent by the MongoDB instance in bytes.
The refresh interval can be changed using the following command:
mongostat [options] [sleep time]
mongotop utility provides you with a mechanism to get information about time spent on read/write operations. This command is similar to Unix's
The following screenshot shows you a simple usage of
mongosniff is a tool that is used to fetch live MongoDB collection statistics. While inserting or querying data from the MongoDB instances, you can run the
mongosniff command and connect it to your MongoDB instance to see what the database does.
Please note that in order to use this utility, you should install the
libpcap library first. To install the
libpcap library, please visit its official website at http://www.tcpdump.org/#documentation.
A simple usage of the
mongosniff tool is as follows:
sudo mongosniff --source NET lo0
The preceding command line will listen to the loopback interface (localhost). This interface is
lo0 in Mac OS systems and
lo for other operating systems, usually. You can get the list for your network interfaces using the
ifconfig command. If you're using Windows as the operating system, you can get the list of network interfaces using the following command:
With the help of GridFS, MongoDB can be used as a filesystem. The processes in this section are used to manage and control the GridFS feature.
There is one process in this category, which is as follows:
This utility enables developers to retrieve files that are stored in the database in the GridFS collection. The
mongofiles utility come in handy when developers need to interact with files stored in the database from the command-line environment.
The usage of this command looks like the following:
mongofiles <options> <commands> <filename>
The following example is a simple usage of this utility:
mongofiles -d mydb list
The preceding command line will retrieve all files in the GridFS collection from the
For more information on
mongofiles, please visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongofiles/.
In this chapter, we went through basic MongoDB topics such as major MongoDB processes, how they work, and why they exist.
We learned that MongoDB consists of some main components such as core, import and export, GridFS, and diagnostic tools. Then, we discussed the basic processes that make MongoDB work, that is,
mongos. Also, we learned that developers or system administrators can manage MongoDB using the
Next, we talked about import and export tools that give developers the ability to export and import objects from and into the database, which is used for backup and restoration procedures. In addition, you can find a brief description of the GridFS components and diagnostic tools that are required to work with the filesystem. We also learned how to find database statistics and issues with diagnostic tools.
In the next chapter, we will learn about the causes of failure in MongoDB and find remedies and solutions to overcome these problems.