To be able to diagnose a MongoDB server or change the default preferences to provide better performance in a database, we need to understand the primitive MongoDB settings and management tools. MongoDB consists of binary files and services that make the infrastructure of the server, and each file performs a specific task.
In this chapter, we will go through the MongoDB processes to discover why they exist and what each process does exactly.
This section contains a brief description of the MongoDB components, file names, and the main purpose of each of them. We will further discuss each item in detail. The following diagram shows you the four main components of MongoDB:
MongoDB components can mainly be classified into the following categories:
Core components
Import and export tools
Diagnostic tools
File storage (GridFS) tools
Each file can be placed into one of the aforementioned categories. The core component files are used to run the MongoDB server and start it. The files are also used to manage the MongoDB server from the command-line interface or manage clustering tasks.
Using import and export tools, developers can create dump files from their database in different formats such as BSON, JSON or CSV and restore them into another database again. Certain tools are used to create or restore BSON files, and some tools are responsible for generating and importing other common formats such as JSON or CSV.
MongoDB has many built-in diagnostic tools to manage and control the currently running server. Finally, we can use GridFS tools to interact with filesystem and GridFS components. In the next section, we will give you more details for each category and the processes inside them.
Inside this category, you can find the core processes that are required to start the MongoDB server. The MongoDB engine uses these tools to accept requests from different clients.
The components and executable files in this group are as follows:
mongodmongomongos
The mongod component is the primary process that is needed for MongoDB to start the server. It manages requests or queries and handles connections.
The following screenshot is the result of executing mongod from the command-line interface:
The preceding screenshot illustrates the result of running mongod from the command line, that is, you finally have a MongoDB server running on port 27017 and a web interface on port 28017, which is the default port.
Like other MongoDB commands, in order to see the parameters of the mongod process, you can simply run following command:
mongod --help
Here, you can see the result of running the mongod command with the --help parameter. This is shown in the following screenshot:
Note
To read more about the command and figure out what each parameter does, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongod/.
After starting the database server, we need to interact with it to issue a command, run commands, run queries, or get reports. In the MongoDB structure, the mongo file is responsible for this task. This is an interactive JavaScript shell that you can utilize from your command-line environment.
Using this executable file, database administrators and also developers can manage the database server or get available databases and collections.
The following screenshot is the result of running the mongo command:
Fortunately, the mongo component provides internal help so that developers can use it to get more information for each command.
The following screenshot shows you the result of running the help command:
Note
To acquire more information about MongoDB and all its parameters, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongo/.
The mongos instance is responsible for routing read/write commands to shards in a sharded cluster. In fact, all processes and applications will connect to this instance and run queries, then the mongos instance will route commands to available shards. The applications will not interact with shards directly. To run the mongos instance, we can provide sharded cluster configurations using a config file or command-line parameters.
In further chapters, we will discuss this process in detail.
Note
To gain more knowledge about the mongos command and its parameters, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongos/.
The tools in this group are used to export or restore files from the database. Mainly, components in this category are classified into two different groups. This includes components that work with binary data or interact with common data formats such as JSON, CSV, and so on.
In order to import or export binary data from and into the database, developers can use tools in this group. We use tools and utilities in this category to create or restore backups for our MongoDB server. In further chapters, we will discuss the backup strategies in detail.
The following are the tools that are available to perform these kinds of tasks:
mongodumpmongorestorebsondumpmongooplog
This process comes in handy when developers or system administrators want to create a dump file from a database in the binary format. This utility and other related tools are useful for MongoDB backup strategies.
The mongodump process is able to retrieve data from either mongod or mongos processes.
Note
For more information on mongodump, visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongodump/.
This process is used to restore and write the binary files that are generated by the mongodump process into the database server. To restore data, mongorestore can establish a new database or use an existing database. Just like the mongodump instance, mongorestore can connect to a mongos instance or it can connect directly to the mongod process.
Note
To read more about mongostore, you can visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongorestore/.
The bsondump process is used to convert the BSON format data to common data formats such as JSON. Essentially, bsondump comes in handy when developers want to convert dump files that are generated by mongodump to human-readable formats.
A very simple usage of this command is shown in the following command line:
bsondump data.bson>data.json
The mongooplog is a utility that duplicates oplog from one server to another server, for instance, in order to perform a migration task. In order to perform the migration operation, mongooplog accepts two parameters, the from and to server addresses.
Note
What is oplog?
The oplog or operation log is a capped collection (a fixed-sized collection) of data that keeps the record of all data altering operations. In further chapters, we will explain this feature in detail.
The following is a simple usage of this command:
mongooplog --from server1 --host server2
The preceding command line will connect to the MongoDB instance of server1 and copy the entire oplog to server2 instance.
In this group, we have two utilities that help us generate or import data in human-readable formats such as JSON or CSV into or from the MongoDB instance. These are mentioned in the following lists:
mongoimportmongoexport
In order to export data in JSON or CSV formats from the MongoDB instance, developers can use this utility.
The following is a simple usage of this command:
mongoexport --db mydb --collection posts --out export.json
The preceding command line will connect to a local instance of MongoDB, retrieve all records from the posts collection of the mydb database in the JSON format, and write all outputs to the export.json file.
The mongoimport utility can help you import the produced export files in JSON, CSV or TSV formats into the MongoDB instance. The export files can be generated from either mongoexport or from other third-party export tools.
The following example is a basic usage of this command:
mongoimport --db mydb --collection posts --file export.json
The preceding command line will import the export.json entries into the posts collection of the mydb database. The same instruction can be used for other data formats using the --type option.
One of the important tools of a database system is diagnostic tools. Fortunately, MongoDB has built-in diagnostic tools that enable developers to diagnose the server or get a brief report from the system.
We have the following utilities placed in this group:
mongostatmongotopmongosniffmongoperf
In the next sections, you can read a brief description of each utility.
This tool produces a brief summary of relevant statistics of the currently running MongoDB instances, either the mongod or mongos instance.
The following screenshot illustrates the output of this tool:
The preceding screenshot shows you the number of queries, update, insert, and delete operations from the database every second.
The following bullet list gives you a brief description for each column:
insert: This refers to the number of insert operations per second.update: This refers to the number of update operations per second.delete: This refers to the number of delete operations per second.getmore: This refers to the number ofgetmoreoperations (that is, theitcommand in mongo shell) per second.command: This refers to the number of executed commands since the lastmongostatcall.flushes: This refers to the number offsyncoperations at the time of the lastmongostatexecution. Thefsyncoperation is a system call that flushes all dirty in-memory pages to the disk.mapped: This refers to the total amount of data mapped in megabytes.vsize: This refers to the amount of virtual memory in megabytes used by the process at the time of the lastmongostatexecution.res: This refers to the amount of resident memory in megabytes used by the process at the time of the lastmongostatexecution.locked: This refers to the percentage of time in a global write lock.idx miss: This refers to the percentage of index access attempts that required a page fault.qr: This refers to the number of clients in the queue that are waiting for read operations.qw: This refers to the number of clients in the queue that are waiting for write operations.ar: This refers to the number of clients that execute read operations.aw: This refers to the number of clients that execute write operations.netIn: This refers to the traffic received by the MongoDB instance in bytes.netOut: This refers to the traffic sent by the MongoDB instance in bytes.
The refresh interval can be changed using the following command:
mongostat [options] [sleep time]
The mongotop utility provides you with a mechanism to get information about time spent on read/write operations. This command is similar to Unix's top command.
The following screenshot shows you a simple usage of mongotop:
The mongosniff is a tool that is used to fetch live MongoDB collection statistics. While inserting or querying data from the MongoDB instances, you can run the mongosniff command and connect it to your MongoDB instance to see what the database does.
Note
Please note that in order to use this utility, you should install the libpcap library first. To install the
libpcap library, please visit its official website at http://www.tcpdump.org/#documentation.
A simple usage of the mongosniff tool is as follows:
sudo mongosniff --source NET lo0
The preceding command line will listen to the loopback interface (localhost). This interface is lo0 in Mac OS systems and lo for other operating systems, usually. You can get the list for your network interfaces using the ifconfig command. If you're using Windows as the operating system, you can get the list of network interfaces using the following command:
ipconfig /all
With the help of GridFS, MongoDB can be used as a filesystem. The processes in this section are used to manage and control the GridFS feature.
There is one process in this category, which is as follows:
mongofiles
This utility enables developers to retrieve files that are stored in the database in the GridFS collection. The mongofiles utility come in handy when developers need to interact with files stored in the database from the command-line environment.
The usage of this command looks like the following:
mongofiles <options> <commands> <filename>
The following example is a simple usage of this utility:
mongofiles -d mydb list
The preceding command line will retrieve all files in the GridFS collection from the mydb database.
Note
For more information on mongofiles, please visit the MongoDB documentation page at http://docs.mongodb.org/manual/reference/program/mongofiles/.
In this chapter, we went through basic MongoDB topics such as major MongoDB processes, how they work, and why they exist.
We learned that MongoDB consists of some main components such as core, import and export, GridFS, and diagnostic tools. Then, we discussed the basic processes that make MongoDB work, that is, mongod and mongos. Also, we learned that developers or system administrators can manage MongoDB using the mongo process. This is an interactive JavaScript shell, which enables developers to run and execute commands, queries, and administration procedures.
Next, we talked about import and export tools that give developers the ability to export and import objects from and into the database, which is used for backup and restoration procedures. In addition, you can find a brief description of the GridFS components and diagnostic tools that are required to work with the filesystem. We also learned how to find database statistics and issues with diagnostic tools.
In the next chapter, we will learn about the causes of failure in MongoDB and find remedies and solutions to overcome these problems.
