You're reading from Mastering MongoDB 7.0 - Fourth Edition

Product typeBook

Published inJan 2024

PublisherPackt

ISBN-139781835460474

Edition4th Edition

Concepts

Databases

Authors (7):

Marko Aleksendrić

Arek Borucki

Leandro Domingues

Malak Abu Hammad

Elie Hannouch

Rajesh Nair

Rachelle Palmer

View More author details

Monitoring and Backup in MongoDB

Effective monitoring and backup are essential when deploying MongoDB in a production environment. They play a crucial role during the development phase, aiding in performance optimization and troubleshooting. Hence, these processes should be established from the outset, not added as an afterthought.

In this chapter, you will explore the operational aspects of MongoDB. You will learn the importance of having a reliable backup strategy, steps to ensure your backups are consistent and accurate, and whether they can be successfully restored when required.

This chapter will cover the following topics:

Importance of monitoring
Key metrics to monitor in MongoDB
MongoDB reporting and monitoring tools
Understanding cluster backups
MongoDB backup methods

Monitoring in MongoDB

Monitoring in MongoDB extends beyond the simple task of keeping an eye on your system's health. It provides a comprehensive lens into the database's intricate behaviors, such as how it processes and responds to various requests, manages memory, or deals with concurrent operations. This deep understanding is crucial for spotting inefficiencies, such as slow-performing queries or suboptimal indexing patterns. Monitoring plays a pivotal role in identifying bottlenecks that can hinder performance, allowing for timely interventions.

Another critical aspect of monitoring is ensuring data integrity. Consistent monitoring can detect anomalies or potential data loss scenarios. By using appropriate monitoring tools and best practices, you can ensure that the database not only functions seamlessly, but also aligns perfectly with the specific needs of your applications and end users.

Monitoring clusters

When designing a software system, you make a lot of...

What should you monitor?

The most crucial metric in MongoDB is memory usage. Like all database systems, MongoDB heavily relies on system memory to boost its performance. Whether you're utilizing the default WiredTiger storage engine configuration or not, it's essential to prioritize memory usage monitoring. Grasping the basics of how computer memory operates can give you an edge when interpreting metrics from your monitoring tools. Let's look at some fundamental concepts associated with computer memory.

Page fault

RAM provides speed but at a premium cost. In contrast, while hard disk drives (HDDs) and solid-state drives (SSDs) are slower, they are affordable and ensure your data remains intact during system or power hiccups. Typically, your data resides on these drives. When executing a query, MongoDB seeks the required data from memory. If it's not there, MongoDB retrieves it from the disk and transfers it to memory. This action is called a page fault since...

Monitoring memory usage in WiredTiger

WiredTiger is MongoDB's default storage engine, known for its efficiency and scalability. When you initiate MongoDB with WiredTiger, it establishes an internal memory cache. How sizable is this cache? By default, it's roughly half of the RAM you have, minus 1 GB. Therefore, if you have 16 GB RAM the internal cache is set to about 7.5 GB. But that's not all the memory MongoDB uses. It also sets aside some memory for tasks such as handling connections and sorting data. MongoDB also takes advantage of any extra memory your system has to further improve its performance.

If you want to check or change the WiredTiger cache settings, you can use the MongoDB Shell. Using the parameter storage.wiredTiger.engineConfig.cacheSizeGB, lets you adjust the cache size.

Note

It's usually best to stick with the default cache size. But if your data is highly compressed, you may consider reducing the cache, say by 10-20% to free up some...

Tracking page faults

Page faults might seem harmless when their numbers are low. But if they start to pile up and cross a certain point, they can slow down your system significantly. This slowdown is more noticeable when you're using traditional HDDs, but it remains a concern even with faster SSDs.

To stay ahead of potential issues with page faults, it's advisable to maintain a test environment that mirrors your live setup. This staging environment lets you push your system to its limits, allowing you to see how many page faults it can handle before performance drops. By comparing the stress test data with the actual page faults in your live system, you can gauge the available margin for optimal performance.

If you're curious about the current page fault status, you can look at the extra_info section:

db.adminCommand({ "serverStatus" : 1 })['extra_info']

serverStatus outputs the following result in mongosh:

{ "note":"fields...

Working set calculations

The working set in MongoDB is a vital concept that pertains to the amount of data and indexes that a database accesses over a specific period. This data should ideally be held in the system's RAM to guarantee optimal performance. When it comes to database performance, understanding your working set is pivotal. While MongoDB is capable of managing vast datasets, the speed of data retrieval is significantly affected by whether the data is accessed directly from the RAM or fetched from a disk. Accessing data from the RAM is exponentially faster, which is why the working set—data that is frequently accessed—should ideally reside in memory.

Why is the working set crucial?

As your MongoDB instance serves requests, it tends to pull the required data into RAM. Over time, the most frequently accessed data—your working set—naturally find its place in memory. If this working set fits comfortably in RAM, your database performance...

MongoDB reporting tools overview

MongoDB offers several integrated reporting methods. This section gives a brief overview of these methods and outlines the specific questions they can address.

Utilities

In MongoDB, utilities are specialized tools designed to offer insights, facilitate diagnostics, and aid in the routine evaluation of database operations. They play a pivotal role in ensuring smooth performance, and assisting administrators in understanding and rectifying any discrepancies or issues. Whether you're looking to get a snapshot of the current activity or dive deep into specific operations, MongoDB's utilities are your go-to resource:

mongostat: This tool provides counts of database operations by type (insert, query, update, delete, etc.), helping you understand the overall server load distribution.
mongotop: This tool tracks and reports current read and write activities on a collection basis. It is particularly useful for identifying which collections...

Hosted monitoring tools overview

Table 12.1 lists various hosted monitoring tools, available typically via a subscription:

MongoDB backup methods

Ensuring data safety and availability is paramount when running MongoDB in a production setting. Several methods can be used to capture and restore backups, providing security against data loss events.

Cloud-based backup solutions

Utilizing cloud database as a service (DBaaS) offerings often simplifies the backup process. For instance, with MongoDB Atlas, backups can be easily managed either through its graphical user interface or its API.

MongoDB Atlas provides the Cloud Backups feature. This service leverages the snapshot capabilities of the respective cloud providers—whether AWS, Azure, or Google Cloud Platform. Notably, it allows the following:

Users can initiate on-demand backups or set up continuous cloud backups. The frequency and retention period of these backups are determined by the specific MongoDB Atlas service tier chosen.
The Cloud Backups feature is exclusive to the platform's paid plans.
The continuous cloud...

Common mistakes and pitfalls in MongoDB monitoring and backup

When managing MongoDB, monitoring and backup are essential. Recognizing potential issues and implementing preventive measures is crucial. Let's explore some common challenges, and tips to sidestep them effectively:

Neglecting replica set status: MongoDB's replica sets offer redundancy and high availability. However, not keeping an eye on the status of your replica sets can lead to potential data loss or availability issues.
- How to avoid it: Regularly check the health and status of each member in your replica set. Set up alerts for any changes in the replica set's status.
Back up without oplog data: The oplog in MongoDB keeps a record of all operations that modify the data. Neglecting to include oplog data in backups can prevent point-in-time restores.
- How to avoid it: Ensure your backup strategy includes oplog data. This allows for point-in- time recovery, which can be crucial for recovering from...

Summary

Monitoring and backup are paramount for MongoDB deployments. Monitoring ensures optimal performance, while backups safeguard against data loss. Various tools and methods, ranging from MongoDB Atlas to filesystem snapshots, provide tailored solutions. Whether in the cloud or on-premises, consistent monitoring and robust backup strategies are essential for maintaining MongoDB's reliability and resilience. In today's rapidly digitalizing world, the ability to efficiently search and analyze data is crucial.

This brings us to the next chapter, Chapter 13, Introduction to Atlas Search. In this chapter, you will explore the powerful search capabilities integrated into MongoDB Atlas, and discover how to harness the full potential of your data through sophisticated search functionalities.

The rest of the chapter is locked

You have been reading a chapter from

Mastering MongoDB 7.0 - Fourth Edition

Published in: Jan 2024Publisher: PacktISBN-13: 9781835460474

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

undefined

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at $15.99/month. Cancel anytime

Authors (7)

Marko Aleksendrić

Marko Aleksendrić is an analyst, an ex-scientist, and a freelance self-taught web developer with over 20 years of experience. Marko has authored the book Modern Web Development with the FARM Stack, published by Packt Publishing. With a keen interest in backend and frontend development, he has been an avid MongoDB user for the last 15 years for various web and data analytics-related projects, with Python and JavaScript as his main tools.
Read more about Marko Aleksendrić

Arek Borucki

Arek Borucki, a recognized MongoDB Champion and certified database administrator, has been working with MongoDB technology since 2016. As principal SRE database engineer, he works closely with technologies such as MongoDB, Elasticsearch, PostgreSQL, Kafka, Kubernetes, Terraform, AWS, and GCP. His extensive experience includes working with renowned companies such as Amadeus, Deutsche Bank, IBM, Nokia, and Beamery. Arek is also a Certified Kubernetes Administrator and developer, an active speaker at international conferences, and a co-author of questions for the MongoDB Associate DBA Exam.
Read more about Arek Borucki

Leandro Domingues

Leandro Domingues is a MongoDB Community Champion and a Microsoft Data Platform MVP alumnus. Specializing in NoSQL databases, focusing on MongoDB, he has authored several articles and is also a speaker and organizer of events and conferences. In addition to teaching MongoDB, he was a professor at one of the largest universities in Brazil. Leandro is passionate about MongoDB and is a mentor and an inspiration to many developers and administrators. His efforts make MongoDB a more comprehensible tool for everyone.
Read more about Leandro Domingues

Malak Abu Hammad

Malak Abu Hammad is a seasoned software engineering manager at Chain Reaction, with a decade of expertise in MongoDB. She has carved a niche for herself not only in MongoDB but also in essential web app technologies. Along with conducting various online and offline workshops, Malak is a MongoDB Champion and a founding member of the MongoDB Arabic Community. Her vision for MongoDB is a future with an emphasis on Arabic localization, aimed at bridging the gap between technology and regional dialects.
Read more about Malak Abu Hammad

Elie Hannouch

Elie Hannouch is a senior software engineer and digital transformation expert. A driving force in the tech industry, he has a proven track record of delivering robust, scalable, and impactful solutions. As a start-up founder, Elie combines his extensive engineering background with strategic innovation to redefine how enterprises operate in today's digital age. Apart from being a MongoDB Champion, Elie leads the MongoDB, Google, and CNCF communities in Lebanon and works toward empowering aspiring tech professionals by demystifying complex concepts and inspiring a new generation of tech enthusiasts.
Read more about Elie Hannouch

Rajesh Nair

Rajesh Nair is a software professional from Kerala, India, with over 12 years of experience working in various MNCs. He started his career as a database administrator for multiple RDBMS technologies, including Progress OpenEdge and MySQL. Rajesh also managed huge datasets for critical applications running on MongoDB as a MongoDB administrator for several years. He has worked on technologies such as MongoDB, AWS, Java, Kafka, MySQL, Progress OpenEdge, shell scripting, and Linux administration. Rajesh is currently based out of Amsterdam, Netherlands, working as a senior software engineer.
Read more about Rajesh Nair

Rachelle Palmer

Rachelle Palmer is the Product Leader for Developer Database Experience and Developer Education at MongoDB, overseeing the driver client libraries, documentation, framework integrations, and MongoDB University. She has built sample applications for MongoDB in Java, PHP, Rust, Python, Node.js, and Ruby. Rachelle joined MongoDB in 2013 and was previously the director of the technical services engineering team, creating and managing the team that provided support and CloudOps to MongoDB Atlas.
Read more about Rachelle Palmer

Personalised recommendations for you

Based on your interests and search pattern

Et al.

Ever wonder why speech recognition systems don't understand the Scottish accent, or what would happen if an astronaut only ate mac 'n' cheese, or other spurious reflections you'd have at a bar? We did, then collated those deliberations into absurd research articles with fake figures and methodologies inspired by even more fictionally absurd studies.

BookAug 2023230 pages5

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages4

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages5

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages1

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages5

Mastering Tableau 2023

This book is a comprehensive resource to mastering your Tableau skills and becoming a BI expert. As you progress, you will learn how to build advanced dashboards and improve your storytelling to derive key business insight, as well as make you well-versed with advanced functionalities of Tableau in the business intelligence domain.

BookAug 2023684 pages

Building AI Applications with ChatGPT APIs

This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.

BookSep 2023258 pages5

Building AI Applications with ChatGPT APIs

This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.

BookSep 2023258 pages2

Data Engineering with AWS

Embark on a journey to master data engineering pipelines on AWS! Our book offers a hands-on experience of AWS services for ingesting, transforming, and consuming data. Whether you're an absolute beginner or someone with basic data engineering experience, this guide is an indispensable resource.

BookOct 2023636 pages5

Modern Data Architecture on AWS

Every organization wants an agile, performant, and cost-effective data platform that meets all their current and future business needs. Purpose-built AWS analytics services and their features play a big part in building such a modern data platform. This book brings to you all the design and architectural patterns that’ll help you achieve this goal.

BookAug 2023420 pages5

Practical Guide to Applied Conformal Prediction in Python

Discover the power of Conformal Prediction with the "Practical Guide to Applied Conformal Prediction in Python." Master the latest techniques to quantify uncertainty in machine learning and computer vision models, and seamlessly apply them to your industry applications.

BookDec 2023240 pages

TinyML Cookbook

With over 70 project-based recipes, the TinyML Cookbook is a practical guide that will help you to get the most out of your microcontrollers. It provides a comprehensive understanding of the theoretical foundations while giving you hands-on experience training ML models for deployment on Arduino Nano 33 BLE Sense, Raspberry Pi Pico, and SparkFun RedBoard Artemis Nano microcontrollers.

BookNov 2023664 pages

Tool name	Description
MongoDB Cloud Manager	A cloud suite for managing MongoDB deployments, providing monitoring, backup, and automation. For on-premises solutions, MongoDB Enterprise Advanced includes Ops Manager.
VividCortex	Provides deep insights into MongoDB's production database workload and query performance with one-second resolution.
Scout	Provides several plugins, including MongoDB Monitoring, MongoDB Slow Queries, and MongoDB Replica Set Monitoring.
Server Density	Provides a MongoDB-specific dashboard, alerts, replication failover timeline, and mobile apps. ...