Scripting in Elasticsearch

Elasticsearch has a powerful way of extending its capabilities by using custom scripts, which can be written in several programming languages. The most common ones are Painless, Express, and Mustache. In this chapter, we will explore how it's possible to create custom scoring algorithms, specially-processed return fields, custom sorting, and complex update operations on records. The scripting concept of Elasticsearch is an advanced stored-procedure system in the NoSQL world; due to this, every advanced user of Elasticsearch should learn how to master it.

Elasticsearch natively provides scripting in Java (that is, a Java code compiled in JAR), Painless, Express, and Mustache; however, a lot of other interesting languages are also available as plugins, such as Kotlin and Velocity. In older Elasticsearch releases, prior to version 5.0...

Painless scripting

Painless is a simple, secure scripting language that is available in Elasticsearch by default. It was designed by the Elasticsearch team to be used specifically with Elasticsearch and can safely be used with inline and stored scripting. Its syntax is similar to Groovy.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

In order to execute the commands, any HTTP client can be used, such as Curl (https://curl.haxx.se/) or Postman (https://www.getpostman.com/). You can use the Kibana console as it provides code completion and better character...

Installing additional script plugins

Elasticsearch provides native scripting (that is, Java code compiled in JAR) and Painless, but a lot of other interesting languages are also available, such as Kotlin.

At the time of writing this book, there are no available language plugins as part of Elasticsearch's official ones. Usually, plugin authors will take up to a week or a month to update their plugins to the new version after a major release.

As previously stated, the official language is now Painless, and this is provided by default in Elasticsearch for better sandboxing and performance.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading...

Managing scripts

Depending on your scripting usage, there are several ways of customizing Elasticsearch in order to use your script extensions.

In this recipe, we will demonstrate how you can provide scripts to Elasticsearch using files, indexes, or in lines.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

In order to execute the commands, any HTTP client can be used, such as Curl (https://curl.haxx.se/) or Postman (https://www.getpostman.com/). You can use the Kibana console as it provides code completion and better character escaping for Elasticsearch.

To...

Sorting data using scripts

Elasticsearch provides scripting support for sorting functionality. In real-world applications, there is often a need to modify the default sorting using an algorithm that is dependent on the context and some external variables. Some common scenarios are as follows:

• Sorting places near a point
• Sorting by most read articles
• Sorting items by custom user logic
• Sorting items by revenue

Because the computing of scores on a large dataset is very CPU-intensive, if you use scripting, then it's better to execute it on a small dataset using standard score queries for detecting the top documents, and then execute a rescoring on the top subset.

Getting ready

You will need an up-and-running Elasticsearch installation...

Computing return fields with scripting

Elasticsearch allows us to define complex expressions that can be used to return a new calculated field value.

These special fields are called script_fields, and they can be expressed with a script in every available Elasticsearch scripting language.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute the commands, any HTTP client can be used, such as Curl (https://curl.haxx.se/) or postman (https://www.getpostman.com/). You can use the Kibana console as it provides code completion and better character...

Filtering a search using scripting

In Chapter 4, Exploring Search Capabilities, we explored many filters. Elasticsearch scripting allows the extension of a traditional filter by using custom scripts.

Using scripting to create a custom filter is a convenient way to write scripting rules that are not provided by Lucene or Elasticsearch, and to implement business logic that is not available in a DSL query.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute the commands, any HTTP client can be used, such as curl (https://curl.haxx.se/) or postman...

Using scripting in aggregations

Scripting can be used in aggregations for extending its analytics capabilities in order to change the values used in metric aggregations or to define new rules to create buckets.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute the commands, any HTTP client can be used, such as curl (https://curl.haxx.se/) or postman (https://www.getpostman.com/). You can use the Kibana console as it provides code completion and better character escaping for Elasticsearch.

To correctly execute the following commands, you will need...

Updating a document using scripts

Elasticsearch allows you to update a document in-place. Updating a document using scripting reduces network traffic (otherwise, you need to fetch the document, change the field or fields, and then send them back) and improves performance when you need to process a large number of documents.

Getting ready

You will need an up-and-running Elasticsearch installation – similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute the commands, any HTTP client can be used, such as curl (https://curl.haxx.se/) or postman (https://www.getpostman.com/). You can use the Kibana console as it provides code...

Reindexing with a script

Reindexing is a new functionality that was introduced in Elasticsearch 5.x for automatically reindexing your data in a new index. This action is often done for a variety of reasons, mainly for mapping changes that require a full reindex of your data.

Getting ready

You will need an up-and-running Elasticsearch installation—similar to the one that we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Managing Mapping.

To execute curl using the command line, then you will need to install curl for your operating system.

In order to correctly execute the following commands, you will need an index that is populated with the chapter_09/populate_for_scripting.sh script...