Backing Up Workloads

Accidents and disasters happen, and just like you may have insurance for these events in real life, you should have insurance for your cluster and workloads.

Most Kubernetes distributions do not include any components to back up workloads, but there are a number of products available from both the open source community and vendor-supported solutions from companies such as Kasten, Veritas, and Commvault.

In this chapter, we will cover the following topics:

• Understanding Kubernetes backups
• Performing an etcd backup
• Introducing and setting up VMware's Velero
• Using Velero to back up workloads
• Managing Velero using the CLI
• Restoring from a backup

To back up your KinD cluster, we will introduce you to a popular open source backup solution called Velero, which can be used to create full backups of workloads and the persistent data in a cluster. We will explain how to use Velero to back up namespaces and objects...

Technical requirements

To perform the hands-on experiments in this chapter, you will need the following:

• A Docker host installed using the steps from Chapter 1, Docker and Container Essentials, with a minimum of 8 GB of RAM
• A KinD cluster configured using the initial scripts from Chapter 2, Deploying Kubernetes Using KinD

You can access the code for this chapter by going to this book's GitHub repository: https://github.com/PacktPublishing/Kubernetes---An-Enterprise-Guide-2E/tree/main/chapter11.

Understanding Kubernetes backups

Backing up a Kubernetes cluster requires backing up not only the workloads running on the cluster but also the cluster itself. Remember that the cluster state is maintained in an etcd database, making it a very important component that you need to back up to recover from any disasters.

Creating a backup of the cluster and the running workloads allows you to do the following:

• Migrate clusters
• Create a development cluster from a production cluster
• Recover a cluster from a disaster
• Recover data from persistent volumes
• Namespace and deployment recovery

In this chapter, we will provide the details and tools to back up your etcd database and every namespace and object in the cluster.

Recovering a cluster from a complete disaster in an enterprise usually involves backing up custom SSL certificates for various components, such as Ingress controllers, load-balancers, and the API server.

Since...

Performing an etcd backup

Since we are using KinD for our Kubernetes cluster, we can create a backup of the etcd database, but we will not be able to restore it.

Our etcd server is running in a pod on the cluster called etcd-cluster01-control-plane, located in the kube-system namespace. During the creation of the KinD cluster, we added an extra port mapping for the control plane node, exposing port 2379, which is used to access etcd. In your own production environment, you may not have the etcd port exposed for external requests, but the process of backing up the database will still be similar to the steps explained in this section.

Backing up the required certificates

Most Kubernetes installations store certificates in /etc/kubernetes/pki. In this respect, KinD is no different, so we can back up our certificates using the docker cp command.

We have included a script in the chapter11/etcd directory called install-etcd-tools.sh that will execute the steps to download...

Introducing and setting up VMware's Velero

Velero is an open source backup solution for Kubernetes that was originally developed by a company called Heptio. As VMware has enhanced their support for Kubernetes, they have purchased multiple companies and Heptio was one of their acquisitions – bringing Velero into the VMware portfolio.

VMware has moved most of its offerings around Kubernetes under the Tanzu umbrella. This can be a little confusing for some people since the original iteration of Tanzu was a deployment of multiple components that added Kubernetes support to vSphere clusters. Since the initial incarnation of Tanzu, it has come to include components such as Velero, Harbor, and the Tanzu Application Platform (TAP), all of which do not require vSphere to function; they will run natively in any standard Kubernetes cluster.

Even with all of the ownership and branding changes, the base functions of Velero have remained. It offers many features that are only...

Using Velero to back up workloads

Velero supports running a "one-time" backup with a single command or on a recurring schedule. Whether you chose to run a single backup or a recurring backup, you can back up all objects or only certain objects using include and exclude flags.

Running a one-time cluster backup

To create an initial backup, you can run a single Velero command that will back up all of the namespaces in the cluster.

Executing a backup without any flags to include or exclude any cluster objects will back up every namespace and all of the objects in the namespace.

To create a one-time backup, execute the velero command with the backup create <backup name> option. In our example, we have named the backup initial-backup:

velero backup create initial-backup

The only confirmation you will receive from this is that the backup request was submitted:

Backup request "initial-backup" submitted successfully.
Run `velero backup...

Managing Velero using the CLI

Right now, all Velero operations must be done using the Velero executable. Managing a backup system without a GUI can be a challenge at first, but once you get comfortable with the Velero management commands, it becomes easy to perform operations.

The Velero executable accepts two options:

• Commands
• Flags

A command is an operation such as backup, restore, install, and get. Most initial commands require a second command to make a complete operation. For example, a backup command requires another command, such as create or delete, to form a complete operation.

There are two types of flags – command flags and global flags. Global flags are flags that can be set for any command, while command flags are specific to the command being executed.

Like many CLI tools, Velero includes built-in help for every command. If you forget some syntax or want to know what flags can be used with a command, you can use the -h flag...

Restoring from a backup

With any luck, you will rarely need to execute a restore of any Kubernetes object.

Even if you haven't been in the IT field long, you have likely experienced a personal situation where you had a drive failure, or accidentally deleted an important file. If you don't have a backup of the data that was lost, it is a very frustrating situation. In the enterprise world, missing data or not having a backup can lead to huge revenue losses, or in some scenarios, large fines in regulated industries.

To run a restore from a backup, you use the create restore command with the --from-backup <backup name> tag.

Earlier in the chapter, we created a single, one-time backup, called initial-backup, which includes every namespace and object in the cluster. If we decided that we needed to restore that backup, we would execute a restore using the Velero CLI:

velero restore create --from-backup initial-backup

The output from the restore command...

Summary

Backing up clusters and workloads is a requirement for any enterprise cluster. In this chapter, we reviewed how to back up the etcd cluster database using etcdctl and the snapshot feature. We also went into detail on how to install Velero in a cluster to back up and restore workloads. We closed out the chapter by copying workloads from an existing backup by restoring an existing backup on a new cluster.

Having a backup solution allows you to recover from a disaster or human error. A typical backup solution allows you to restore any Kubernetes object, including namespaces, persistent volumes, RBAC, services, and service accounts. You can also take all of the workloads from one cluster and restore them on a completely different cluster for testing or troubleshooting.

Coming up in the next chapter, we will introduce you to Istio, a popular open source service mesh.

Questions

1. True or false – Velero can only use an S3 target to store backup jobs.
   1. True
   2. False
2. If you do not have an object storage solution, how can you provide an S3 target using a backend storage solution such as NFS?
   1. You can't – there is no way to add anything in front of NFS to present S3.
   2. Kubernetes can do this using native CSI features.
   3. Install MinIO and use the NFS volumes as persistent disks in the deployment.
   4. You don't need to use an object store; you can use NFS directly with Velero.
3. True or false – Velero backups can only be restored on the same cluster where the backup was originally created.
   1. True
   2. False
4. What utility can you use to create an etcd backup?
   1. Velero.
   2. MinIO.
   3. There is no reason to back up the etcd database.
   4. etcdctl.
5. Which...