Chapter 16: Plugin Development

Elasticsearch is designed to be extended with plugins to improve its capabilities. In the previous chapters, we installed and used many of them (new queries, REST endpoints, and scripting plugins).

Plugins are application extensions that can add many features to Elasticsearch. They can have several usages, including the following:

• Adding a new scripting language (that is, Python and JavaScript plugins)
• Adding new aggregation types
• Adding a new ingest processor
• Extending Lucene-supported analyzers and tokenizers
• Using native scripting to speed up the computation of scores, filters, and field manipulation
• Extending node capabilities, for example, creating a node plugin that can execute your logic
• Monitoring and administering clusters

In this chapter, the Java language will be used to develop a native plugin, but it is possible to use any Java virtual machine (JVM) language that generates JAR files.

The standard...

Creating a plugin

Native plugins allow several aspects of the Elasticsearch server to be extended, but they require good knowledge of Java.

In this recipe, we will see how to set up a working environment to develop native plugins.

Getting ready

You need an up and running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

Gradle or an integrated development environment (IDE) that supports Java programming with Gradle (version 7.3.x used in the examples), such as Eclipse, Visual Studio Code, or IntelliJ IDEA, is required. Java JDK 17 or above needs to be installed.

The code for this recipe is available in the ch16/simple_plugin directory.

How to do it...

Generally, Elasticsearch plugins are developed in Java using the Gradle build tool (https://gradle.org/) and deployed as a ZIP file.

To create a simple JAR plugin, we will perform the following steps:

1. To correctly build and...

Creating an analyzer plugin

Elasticsearch provides a large set of analyzers and tokenizers to cover general needs out of the box. Sometimes, we need to extend the capabilities of Elasticsearch by adding new analyzers.

Typically, you can create an analyzer plugin when you need to do the following:

• Add standard Lucene analyzers/tokenizers that are not provided by Elasticsearch.
• Integrate third-party analyzers.
• Add custom analyzers.

In this recipe, we will add a new custom English analyzer, similar to the one provided by Elasticsearch.

Getting ready

You need an up and running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

Gradle or an integrated development environment (IDE) that supports Java programming with Gradle (version 7.3.x used in the examples), such as Eclipse, Visual Studio Code, or IntelliJ IDEA, is required. Java JDK 17 or above needs to be installed.

The...

Creating a REST plugin

In the previous recipe, we read how to build an analyzer plugin that extends the query capabilities of Elasticsearch. In this recipe, we will see how to create one of the most common Elasticsearch plugins. This kind of plugin allows the standard REST calls to be extended with custom ones to easily improve the capabilities of Elasticsearch.

In this recipe, we will see how to define a REST entry point and create its action; in the next one, we'll see how to execute this action distributed in shards.

Getting ready

You need an up and running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

Gradle or an integrated development environment (IDE) that supports Java programming with Gradle (version 7.3.x used in the examples), such as Eclipse, Visual Studio Code, or IntelliJ IDEA, is required. Java JDK 17 or above needs to be installed.

The code for this recipe is available...

Creating a cluster action

In the previous recipe, we saw how to create a REST entry point, but to execute the action at the cluster level, we will need to create a cluster action.

An Elasticsearch action is generally executed and distributed in the cluster and, in this recipe, we will see how to implement this kind of action. The cluster action will be very bare; we send a string with a value to every shard and the shards echo a result string, concatenating the string with the shard number.

Getting ready

You need an up and running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

Gradle or an integrated development environment (IDE) that supports Java programming with Gradle (version 7.3.x used in the examples), such as Eclipse, Visual Studio Code, or IntelliJ IDEA, is required. Java JDK 17 or above needs to be installed.

The code for this recipe is available in the ch16/rest_plugin directory...

Creating an ingest plugin

Elasticsearch 5.x introduced the ingest node that allows the modification, via a pipeline, to the records before ingesting in Elasticsearch. We have already seen in Chapter 12, Using the Ingest Module, that a pipeline is composed of one or more processor actions. In this recipe, we will see how to create a custom processor that stores in a field the initial character of another one.

Getting ready

You need an up and running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

Gradle or an integrated development environment (IDE) that supports Java programming with Gradle (version 7.3.x used in the examples), such as Eclipse, Visual Studio Code, or IntelliJ IDEA, is required. Java JDK 17 or above needs to be installed.

The code for this recipe is available in the ch16/ingest_plugin directory.

How to do it...

To create an ingest processor plugin, we need to create the...