Preventing Disasters

In an enterprise network, a disaster can strike at any time. While, as administrators, we always do our best to design the most stable and fault-tolerant server implementations we possibly can, what matters most is how we are able to deal with disasters when they do happen. As stable as server hardware is, any component of a server can fail at any time. In the face of a disaster, we need a plan. How can you attempt to recover data from a failed disk? What do you do when your server all of a sudden decides it doesn’t want to boot? These are just some of the questions we’ll answer as we take a look at several ways we can prevent and recover from disasters. In our final chapter, we’ll cover the following topics:

• Preventing disasters
• Utilizing Git for configuration management
• Implementing a backup plan
• Utilizing bootable recovery media

We’ll start off our final chapter by looking at a few tips to help...

Preventing disasters

As we proceed through this chapter, we’ll look at ways we can recover from disasters. However, if we can prevent a disaster from occurring in the first place, then that’s even better. We certainly can’t prevent every type of disaster that could possibly happen but having a good plan in place and following that plan will lessen the likelihood. A good disaster recovery plan will include a list of guidelines to be followed with regard to implementing new servers and managing current ones.

This plan may include information such as an approved list of hardware (such as hardware configurations known to work efficiently in an environment), as well as rules and regulations for users, a list of guidelines to ensure physical and software security, proper training for end users, and method change control. Some of these concepts we’ve touched on earlier in the book but are worth repeating from the standpoint of disaster prevention.

First...

Utilizing Git for configuration management

One of the most valuable assets on a server is its configuration. This is second only to the data the server stores. Often, when we implement new technology on a server, we’ll spend a great deal of time editing configuration files all over the server to make it work as best as we can. This can include any number of things, from Apache virtual host files to DHCP server configuration, DNS zone files, and more. If a server were to encounter a disaster from which the only recourse was to completely rebuild it, the last thing we’d want to do is re-engineer all of this configuration from scratch. This is where Git comes in.

In a typical development environment, an application being developed by a team of engineers can be managed by Git, each contributing to a repository that hosts the source code for their software. One of the things that makes Git so useful is how you’re able to go back to previous versions of a file in...

Implementing a backup plan

Creating a solid backup plan is one of the most important things you’ll ever do as a server administrator. Even if you’re only using Ubuntu Server at home as a personal file server, backups are critical. During my career, I’ve seen disks fail many times. I often hear arguments about which hard disk manufacturer beats others in terms of longevity, but I’ve seen disk failures so often, I don’t trust any of them. All disks will fail eventually, it’s just a matter of when. And when they do fail, they’ll usually fail hard with no easy way to recover data from them. A sound approach to managing data is that any disk or server can fail, and it won’t matter, since you’ll be able to regenerate your data from other sources, such as a backup or secondary server.

There’s no one best backup solution, since it all depends on what kind of data you need to secure, and what software and hardware resources...

Utilizing bootable recovery media

The concept of live media is a wonderful thing, as we can boot into a completely different working environment from the operating system installed on our device and perform tasks without disrupting installed software on the host system. The desktop version of Ubuntu, for example, offers a complete computing environment we can use in order to not only test hardware and troubleshoot our systems but also browse the web just as we would on an installed system. In terms of recovering from disasters, live media becomes a saving grace.

As administrators, we run into one problem after another. This gives us our job security. Computers often seemingly have a mind of their own, failing when least expected (as well as seemingly every holiday). Our servers and desktops can encounter a fault at any time, and live media allows us to separate hardware issues from software issues, by troubleshooting from a known good working environment.

One of my favorites...

Summary

In this chapter, we looked at several ways in which we can prevent and recover from disasters. Having a sound prevention and recovery plan in place is an important key to managing servers efficiently. We need to ensure we have backups of our most important data ready for whenever servers fail, and we should also keep backups of our most important configurations. Ideally, we’ll always have a warm site set up with preconfigured servers ready to go in a situation where our primary servers fail, but one of the benefits of open source software is that we have a plethora of tools available to us that we can use to create a sound recovery plan. In this chapter, we looked at leveraging rsync as a useful utility for creating differential backups, and we also looked into setting up a Git server we can use for configuration management, which is also a crucial aspect of any sound prevention plan. We also talked about the importance of live media in diagnosing issues.

And with...

Further reading

• Pro Git by Scott Chacon and Ben Straub: https://learnlinux.link/git-book
• Introduction to RAID terminology and concepts: https://learnlinux.link/raid-concepts

Join our community on Discord

Join our community’s Discord space for discussions with the author and other readers:

https://packt.link/LWaZ0