Chapter 7. Snapshot and Restore

In the previous chapter, we looked at some memory configuration and tips of mapping definition for improving indexing performance. We talked about segments and merging policies. We tried to explain the store module and how to throttle disk I/O operations. Then, we discussed the bulk API. In this chapter, we will look at how can we back up our data and restore. For this purpose, we will examine Elasticsearch snapshot and restore module. By the end of this chapter, we would have covered the following topics:

• How to create a snapshot repository

• How to create a snapshot

• How to restore a snapshot

• How the snapshot process works

Snapshot repository

When working with large amounts of data, backup and restore is an important requirement. Elasticsearch has a snapshot and restore module so that they meet the needs of users for backing up and restoring existing indices. Because Elasticsearch needs to know where to back up data, before backup and restore operations of the indices, a snapshot repository should be registered in Elasticsearch. The following is an example of how to register a snapshot repository:

curl -XPUT localhost:9200/_snapshot/my_backup -d '{
    "type": "fs",
    "settings": {
        "location": "/data/backups/my_backup",
        "compress": true,
        "chunk_size": "10m"
    }
}'

The preceding command registers a shared filesystem repository named as my_backup. It will use location of /data/backups/my_backup. The _snapshot is a REST endpoint for snapshot operations. Its first parameter is the repository name. Repository name will be specified by this parameter and must be unique. The type parameter...

Snapshot

A snapshot is a backup of your cluster index(s). Snapshots are stored in a repository that has been registered before. A repository can contain multiple snapshots of the same cluster. A snapshot can be created by executing the following command:

curl -XPUT localhost:9200/_snapshot/my_backup/first_snapshot

The _snapshot is REST endpoint for snapshot operations. Its second parameter is a unique snapshot name. The preceding command creates a snapshot of all open and started indices in the cluster. If you want to back up a certain index, you must specify the list of indices in the body of the request:

curl -XPUT localhost:9200/_snapshot/my_backup/first_snapshot -d '{
  "indices": "my_index",
  "ignore_unavailable": "true",
  "include_global_state": false,
  "partial": true
}'

The indices parameter supports multi-index syntax; that means you can separate the index names with commas or you can use a wildcard, like *. Default value of the ignore_unavailable parameter is true. It will cause...

Restore

A snapshot can be restored using the following command:

curl -XPOST localhost:9200/_snapshot/my_backup/first_snapshot/_restore

The _restore is the REST endpoint for restore operations. It restores the snapshot mentioned in the previous parameter. The preceding command will restore all indices of the specified snapshot name. If you want to restore just certain indices, you must be specify the list of indices in the body of the request:

curl -XPOST localhost:9200/_snapshot/my_backup/first_snapshot/_restore -d '{
  "indices": "my_index",
  "ignore_unavailable": "true",
  "include_global_state": false,
  "include_aliases": false,
  "partial": true,
  "rename_pattern": "my_(.+)",
  "rename_replacement": "restored_$1"
}'

The indices parameter defines the index names that need to be restored. It supports multi-index syntax. For example, when the * character is used, Elasticsearch will restore all indices of the specified snapshot. If you do not want to restore the aliases, the include_aliases...

How does the snapshot process works?

As stated earlier, a repository can contain multiple snapshots of the same cluster. Therefore, the snapshots files are stored in compact form. This means that your data will not be repeated when you have multiple snapshots of the same indices. At first, Elasticsearch checks the list of the index files. Then, it copies only newly created or changed files since the last snapshot. Now look at the following example:

curl -XGET localhost:9200/my_index/_search?pretty
{
   "took": 3,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 1,
      "hits": [
         {
            "_index": "my_index",
            "_type": "snapshot",
            "_id": "AVCmN4l-7pWKrBPkopj3",
            "_score": 1,
            "_source": {
               "title": "Document A"
            }
         },
         {
            "_index": "my_index",
            "_type": "snapshot"...

Summary

In this chapter, we looked at how to back up and restore our data. For this reason, we examined the snapshot repository, and then we looked at how to create a snapshot and restore it. We talked about their configuration details. Finally, we discussed how snapshot process works. In the next chapter, we will examine how to improve the user search experience. We will look at how you can correct spelling mistakes and we will examine the suggest API. Finally, we will discuss the improving query relevance and we will look at some relevant cases related to this topic.