Exploring Search Capabilities

Now that we have set the mappings and put the data in the indices, we can start exploring the search capabilities in Elasticsearch. In this chapter, we will cover searching using different factors: sorting, highlighting, scrolling, suggesting, counting, and deleting. These actions are the core part of Elasticsearch; ultimately, everything in Elasticsearch is about serving the query and returning good-quality results.

This chapter is divided into two parts: the first part shows how to perform an API call-related search, and the second part will look at two special query operators that are the basis for building complex queries in the following chapters.

In this chapter, we will cover the following recipes:

• Executing a search
• Sorting results
• Highlighting results
• Executing a scrolling query
• Using the search_after functionality
• Returning inner hits in...

Technical requirements

All the recipes in this chapter require us to prepare and populate the required indices—the online code is available on the PacktPub website or via GitHub (https://github.com/aparo/elasticsearch-7.x-cookbook). Here, you can find scripts to initialize all the required data.

Executing a search

Elasticsearch was born as a search engine; its main purpose is to process queries and give results as fast as possible. In this recipe, we'll see that a search in Elasticsearch is not only limited to matching documents - it can also calculate additional information that's required to improve the search quality.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands any, HTTP client can be used such, as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or something similar. I suggest using the Kibana console as it provides...

Sorting results

When searching for results, the standard criterion for sorting in Elasticsearch is the relevance to a text query. Real-world applications often need to control the sorting criteria in scenarios, such as the following:

• Sorting a user by last name and first name
• Sorting items by stock symbols, price (ascending and descending)
• Sorting documents by size, file type, source, and so on
• Sorting item related maximum or minimum or average of some children fields

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl...

Highlighting results

Elasticsearch performs a good job of finding matching results in big text documents. It's useful for searching text in very large blocks, but to improve user experience, you need to show users the abstract—a small portion of the text part of the document that has matched the query. The abstract is a common way to help users understand how the matched document is relevant to them.

The highlight functionality in Elasticsearch is designed to do this job.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl...

Executing a scrolling query

Every time a query is executed, the results are calculated and returned to the user in real time. In Elasticsearch, there is no deterministic order for records—pagination on a big block of values can bring inconsistency between results due to added and deleted documents, and also documents with the same score.

The scrolling query tries to resolve this kind of problem, giving a special cursor that allows the user to uniquely iterate all the documents. 

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such...

Using the search_after functionality

Elasticsearch standard pagination using from and size perform very poorly on large datasets because, for every query, you need to compute and discard all the results before the from value. Scrolling doesn't have this problem, but it consumes a lot due to memory search contexts, so it cannot be used for frequent user queries.

To bypass these problems, Elasticsearch 5.x and above provides the search_after functionality, which provides a fast skipping for scrolling results.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client...

Returning inner hits in results

In Elasticsearch, when using nested and child documents, we can have complex data models. Elasticsearch, by default, returns only documents that match the searched type and not the nested or children one that matches the query.

The inner_hits function was introduced in Elasticsearch 5.x to provide this functionality.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console as it provides code completion...

Suggesting a correct query

It's very common for users to commit typing errors or to require suggestions for words that they are writing. These issues are solved by Elasticsearch with the suggested functionality.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console as it provides code completion and better character escaping for Elasticsearch.

To correctly execute the following commands, you will need an index populated...

Counting matched results

It is often required to return only the count of the matched results and not the results themselves.

There are a lot of scenarios involving counting, such as the following:

• To return the number of something (how many posts for a blog, how many comments for a post).
• Validating whether some items are available. Are there posts? Are there comments?

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console...

Explaining a query

When executing searches, it's very common to have documents that don't match the query as expected. To easily debug these scenarios, Elasticsearch provides the explain query call, which allows you to check how the scores are computed against a document.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console as it provides code completion and better character escaping for Elasticsearch...

Query profiling

This feature is available from Elasticsearch 5.x or above via the profile API. This allows the user to track the time spent by Elasticsearch in executing a search or an aggregation.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console as it provides code completion and better character escaping for Elasticsearch.

To correctly execute the following commands, you will need an index populated with the...

Deleting by query

We saw how to delete a document in the Deleting a document in recipe Chapter 3, Basic Operations. Deleting a document is very fast, but it requires knowing the document ID for direct access, and in some cases, the routing value too.

Elasticsearch provides a call to delete all the documents that match a query using an additional module called re-index, which is installed by default.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/)...

Updating by query

In the previous chapter, we saw how to update a document in the Update a document recipe.

The update_by_query API call allows the user to execute the update on all the documents that match a query. It is very useful if you need to do the following:

• Reindex a subset of your records that match a query. It's common if you change your document mapping and need the documents to be reprocessed.
• Update values of your records that match a query.

It's the Elasticsearch version of the SQL update command.

This functionality is provided by an additional module called reindex that is installed by default. 

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading...

Matching all the documents

One of the most common queries in the match_all query. This kind of query allows the user to return all the documents that are available in an index. The match_all and other query operators are part of the Elasticsearch query DSL.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console as it provides code completion and better character escaping for Elasticsearch.

To correctly execute the following...

Using a Boolean query

Most people using a search engine have, at some time or another, used the syntax with minus (-) and plus (+) to include or exclude query terms. The Boolean query allows the user to programmatically define queries to include, exclude, optionally include (should), or filter in the query.

This kind of query is one of the most important ones because it allows the user to aggregate a lot of simple queries or filters that we will see in this chapter to build a big complex one.

Two main concepts are important in searches: query and filter. The query means that the matched results are scored using an internal Lucene scoring algorithm; for the filter, the results are matched without scoring. Because the filter doesn't need to compute the score, it is generally faster and can be cached.

...

Using the search template

Elasticsearch provides the capability of providing a template and some parameters to fill it. This functionality is very useful because it allows you to manage query templates stored in the .scripts index and allows you to change them without changing the application code.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

To execute these commands, any HTTP client can be used, such as curl (https://curl.haxx.se/), postman (https://www.getpostman.com/), or similar. I suggest using the Kibana console as it provides code completion and better character escaping for...