Search APIs

So far, what we have learned from the preceding four chapters is preparing us for searching. We prepared the index settings, the mappings, the analyzers, and then we indexed the documents. One of the ultimate goals is to execute the query and get high-quality results from the index. The Search API allows you to perform a search query and return the search hits that match the query. There are two basic ways to run a search: one is to send the search request with URL parameters, and the other is to use a request body. This chapter also provides additional information about searching related APIs, such as tuning, validating, and troubleshooting. By the end of this chapter, you will have covered the following topics:

• The URI search
• The Request body search
• The Query domain-specific language (DSL)
• The Multi-search API
• The Explain API
• The Validate API
• The Count API
• The Field...

Indexing sample documents

You will recall that we have introduced 314 exchange-traded funds (ETFs) provided by TD Ameritrade. We are going to index those documents in this chapter. In our GitHub repository (https://github.com/PacktPublishing/Mastering-Elasticsearch-7.0/tree/master/Chapter6), you can download two files, cf_etf_list_bulk.json and cf_etf_list_bulk_index.sh. You need to make the bash file runnable and then you can run it to index those documents. The commands to issue are specified here. Before you issue the following command, make sure your cf_etf index is recreated with the static mappings and the custom analyzer. If not, delete the index and recreate it:

$chmod +x cf_etf_list_bulk_index.sh
$./cf_etf_list_bulk_index.sh

After the command runs successfully, we can use the Postman API to issue the count API to verify the number of indexed documents. The total number...

Search APIs

For simple searches, you can use the simple query string as a URL parameter to execute the Search API. For complex queries, you can write JSON query strings using DSL in the request body. All Search APIs can be applied to multiple indices. Let's show three search examples, with one index, two indices, and all indices. The following screenshot shows how to search all documents with the cf_etf index. 314 documents are identified:

The following screenshot shows how to search all documents with two indices, the cf_etf index, and cf_view index. 316 documents are identified:

The following screenshot shows how to search all documents with all indices. 318 documents are identified:

We will first introduce the URI search and then the request body search in a separate subsection.

...

Query DSL

Query DSL is a JSON-based search language for querying specific and analytical datasets. Basically, a query can be classified in two contexts—a query context and a filter context. In the filter context, the search results are based on the question relating to the match or lack of match to the query clause, where the no scoring value, _score, is provided in the results. In the context of the query, the search results are based on measuring the appropriateness of the match on the query clause. We will introduce a number of common query subtypes in the following subsection.

Full text queries

This type of query will use analyzers (or given search analyzers) to tokenize the query string first. Then, it uses the...

The multi-search API

Similar to the Bulk API, Elasticsearch provides you with a way to gather multiple search (_msearch) requests together to send out in a batch. You need to put the header and the corresponding request body as a pair in the multi-search request body. You can specify the appropriate index names and parameters in the header, including search_type, preference, and routing. The header content type must be set to the newline delimited JSON format (application/x-ndjson).

Now, let's look at an example to issue an _msearch request that contains two sub-requests to find those EFTs in the cf_etf and cf_view indices that do not have the rating field. In the following screenshot, you can see that the response body has a responses field with the array type that contains two results corresponding to the two sub-requests:

...

Other search-related APIs

Elasticsearch supports a few search-related APIs to provide information to troubleshoot search issues. It can help you in preparing the complex search query to obtain more precise search results.

The _explain API

Similar to the explain parameter in the request body search, this _explain API tells you how to calculate the score of matching for a particular document. It provides you with useful feedback when looking for related issues and finding the reason why a given document can be found. The following screenshot provides an explanation as to why the GgOJ7msB3gfI-VGlT4dA document identifier satisfies the query:

...

Summary

What a lengthy chapter! Our excuse is the slogan of Elasticsearch—You know, for Search. Elasticsearch supports many features, and the search functionality is one of the most important topics. We have experienced different types of searches, from term-based to full-text; from exact search to fuzzy search; from a single field search to a multi-search, and then to a compound search. In short, we have explored every avenue from a simple query to a complex query. We also learned a lot about the query DSL and search-related APIs.

In the next chapter, we will focus on data modeling techniques, and some of the most common problems in data modeling and their solutions involving a variety of techniques, including the use of nested objects and parent-child relationships to process related documents. Let's see how we will resolve the data modeling issue when Elasticsearch...