Managing Mapping

Mapping is a very important concept in Elasticsearch, as it defines how the search engine should process a document and its fields.

Search engines perform the following two main operations:

• Indexing: This is the action to receive a document and to process it and store it in an index
• Searching: This is the action to retrieve the data from the index

These two parts are strictly connected; an error in the indexing step leads to unwanted or missing search results.

Elasticsearch has explicit mapping on an index level. When indexing, if a mapping is not provided, a default one is created, and guesses the structure from the data fields that the document is composed of. This new mapping is then automatically propagated to all cluster nodes.

The default type mapping has sensible default values, but when you want to change their behavior or customize several other aspects...

Using explicit mapping creation

If we consider the index as a database in the SQL world, mapping is similar to the table definition.

Elasticsearch is able to understand the structure of the document that you are indexing (reflection) and create the mapping definition automatically (explicit mapping creation).

Getting ready

To execute the code in this recipe, you need an up-and-running Elasticsearch installation, as described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or other similar platforms. I suggest using the Kibana console to provide...

Mapping base types

Using explicit mapping makes it possible to be faster in starting to ingest the data using a schema-less approach without being concerned about field types. Thus, to achieve better results and performance in indexing, it's required to manually define a mapping.

Fine-tuning mapping brings some advantages, such as the following:

• Reducing the index size on the disk (disabling functionalities for custom fields)
• Indexing only interesting fields (general speed up)
• Precooking data for fast search or real-time analytics (such as facets)
• Correctly defining whether a field must be analyzed in multiple tokens or considered as a single token

Elasticsearch allows you to use base fields with a wide range of configurations.

Getting ready

...

Mapping arrays

An array or multi-value fields are very common in data models (such as multiple phone numbers, addresses, names, aliases, and so on), but not natively supported in traditional SQL solutions.

In SQL, multi-value fields require the creation of accessory tables that must be joined to gather all the values, leading to poor performance when the cardinality of records is huge.

Elasticsearch, which works natively in JSON, provides support for multi-value fields transparently.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl...

Mapping an object

An object is a base structure (analogous to a record in SQL). Elasticsearch extends the traditional use of objects, thus allowing for recursive embedded objects.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute the commands, any HTTP client can be used such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. Again, I suggest using Kibana console, which provides code completion and better character escaping for Elasticsearch.

How to do it...

...

Mapping a document

The document is also referred to as the root object. This has special parameters that control its behavior, which are mainly used internally to do special processing, such as routing or time-to-live of documents.

In this recipe, we'll take a look at these special fields and learn how to use them.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, every HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console, which provides code completion and better character escaping...

Using dynamic templates in document mapping

In the Using explicit mapping creation recipe, we have seen how Elasticsearch is able to guess the field type using reflection. In this recipe, we'll see how we can help it improve its guessing capabilities via dynamic templates.

The dynamic template feature is very useful. For example, it may be useful in situations in which you need to create several indices with similar types because it allows you to move the need to define mappings from coded initial routines to automatic index-document creation. A typical usage is to define types for Logstash log indices.

Getting ready

You need an up-and-running Elasticsearch installation as we described in the Downloading and installing...

Managing nested objects

There is a special type of embedded object: the nested one. This resolves a problem related to Lucene indexing architecture, in which all the fields of embedded objects are viewed as a single object. During search, in Lucene, it is not possible to distinguish between values and different embedded objects in the same multi-valued array.

If we consider the previous order example, it's not possible to distinguish an item name and its quantity with the same query, as Lucene puts them in the same Lucene document object. We need to index them in different documents and then join them. This entire trip is managed by nested objects and nested queries.

Getting ready

You need an up-and-running Elasticsearch installation...

Managing a child document with a join field

In the previous recipe, we have seen how it's possible to manage relations between objects with the nested object type. The disadvantage of nested objects is their dependence from their parent. If you need to change a value of a nested object, you need to reindex the parent (this brings a potential performance overhead if the nested objects change too quickly). To solve this problem, Elasticsearch allows you to define child documents.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl...

Adding a field with multiple mappings

Often a field must be processed with several core types or in different ways. For example, a string field must be processed as tokenized for search and not-tokenized for sorting. To do this, we need to define a fields multifield special property.

The fields property is a very powerful feature of mappings because it allows you to use the same field in different ways.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar...

Mapping a GeoPoint field

Elasticsearch natively supports the use of geolocation types—special types that allow you to localize your document in geographic coordinates (latitude and longitude) around the world.

There are two main types that are used in the geographic world: the point and the shape. In this recipe, we'll look at GeoPoint—the base element of geo location.

Getting ready

 

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest...

Mapping a GeoShape field

An extension to the concept of point is the shape. Elasticsearch provides a type that facilitates the management of arbitrary polygons in GeoShape.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To be able to use advanced shape management, Elasticsearch requires two JAR libraries in its classpath (usually the lib directory), as follows:

• Spatial4J (v0.3)
• JTS (v1.13)

How to do it...

To map a geo_shape type, a user must explicitly provide some...

Mapping an IP field

Elasticsearch is used in a lot of systems to collect and search logs, such as Kibana (https://www.elastic.co/products/kibana) and LogStash (https://www.elastic.co/products/logstash). To improve search when using IP addresses, Elasticsearch provides the IPv4 and IPv6 type that can be used to store IP addresses in an optimized way.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

How to do it...

You need to define the type of the field that contains...

Mapping an alias field

It is very common to have a lot of different types in several indices. Because Elasticsearch makes it possible to search in many indices, you should filter for common fields at the same time.

In the real world, these fields are not always called in the same way in all mappings (generally because they are derived from different entities), it's very common to have a mix of added_date, timestamp, @timestamp, and date_add fields that are referring to the same date concept.

The alias fields allow you to define an alias name to be resolved, as well as a query time to simplify the call of all fields with the same meaning.

Getting ready

 

You need an up-and-running Elasticsearch installation...

Mapping a Percolator field

The Percolator is a special type field that makes it possible to store an Elasticsearch query inside the field and use it in percolator query.

The Percolator can be used to detect all queries that match a document.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using Kibana console, which provides code completion and better character escaping for Elasticsearch.

...

Mapping feature and feature vector fields

It's common to have the requirement to score a document dynamically, depending on the context. For example, scoring more particular documents that are inside a category—the classic scenario is to boost (increase low scored) documents that are based on value such as page rank, hits, or categories.

Elasticsearch 7.x provides two new ways to boost your scores based on values: one is the feature fields and the other is its extension to a vector of values.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client...

Adding metadata to a mapping

Sometimes, when we are working with our mapping, it is required to store some additional data to be used for display purposes, ORM facilities, permissions, or simply to track them in the mapping.

Elasticsearch allows you to store every kind of JSON data you want in the mapping with the special _meta field.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

How to do it...

The _meta mapping field can be populated with any data we want. Consider...

Specifying different analyzers

In the previous recipes, we have looked at how to map different fields and objects in Elasticsearch, and we have described how it's easy to change the standard analyzer with the analyzer and search_analyzer properties.

In this recipe, we will loot at several analyzers and learn how to use them to improve indexing and searching quality.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

How to do it...

Every core type field allows...

Mapping a completion field

To be able to provide search functionalities for our user, one of the most common requirements is to provide a text suggestion for our query.

Elasticsearch provides a helper for archiving this functionality using a special type mapping called completion.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

How to do it...

The definition of a completion field is similar to the previous core type fields. For example, to provide a suggestion...