In this chapter, we will cover the following recipes:
Executing a search
Sorting results
Highlighting results
Executing a scrolling query
Using the search_after functionality
Returning inner hits in results
Suggesting a correct query
Counting matched results
Explaining a query
Query profiling
Deleting by query
Updating by query
Matching all the documents
Using a Boolean query
Now we have set the mappings and put the data in the indices, we can search.
In this chapter, we will cover the search using different factors: sorting, highlighting, scrolling, suggesting, counting, and deleting. These actions are the core part of Elasticsearch: ultimately, everything in Elasticsearch is about serving the query and returning good-quality results.
This chapter is divided in two parts: the first part shows how to perform an API call-related search, the last part will look at two special query operators that are the basis for building complex queries in the next chapters.
All the recipes in this chapter require us to prepare and populate the required indices: the online code is available on the PacktPub website (https://www.packtpub.com/big-data-and-business-intelligence/elasticsearch-cookbook) or via GitHub (https://github.com/aparo/elasticsearch-cookbook-third-edition). There are scripts to initialize all the required data.
Elasticsearch was born as a search engine: its main purpose is to process queries and give results.
In this recipe, we'll see that a search in Elasticsearch is not only limited to matching documents, but it can also calculate additional information required to improve the search quality.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
The mapping used in all these chapter queries and searches is the following:
{
"mappings": {
"test-type": {
"properties": {
"pos": {
"type": "integer",
"store": "yes"
},
"uuid": {
...When searching for results, the most common criterion for sorting in Elasticsearch is the relevance to a text query.
Real-world applications often need to control the sorting criteria in scenarios, such as the following:
Sorting a user by last name and first name
Sorting items by stock symbols, price (ascending, descending)
Sorting documents by size, file type, source
You need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via the command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
Elasticsearch performs a good job of finding matching results also in big text documents. It's useful for searching text in very large blocks, but to improve user experience, you need to show users the abstract: a small portion of the text that has matched the query. The abstract is a common way to help users to understand how the matched document is relevant to them.
The highlight functionality in Elasticsearch is designed to do this job.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
Every time a query is executed, the results are calculated and returned to the user. In Elasticsearch, there is not a deterministic order for records: pagination on a big block of values can bring inconsistency between results due to added and deleted documents and also documents with the same score. The scrolling query tries to resolve this kind of problem, giving a special cursor that allows the user to uniquely iterate all the documents.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
Elasticsearch standard pagination using from and size performs very poorly on large datasets because for every query you need to compute and discard all the results before the from value. The scrolling doesn't have this problem, but it consumes a lot, due to memory search contexts, so it cannot be used for frequent user queries.
To bypass these problems, Elasticsearch 5.x provides the search_after functionality that provides a fast skipping for scrolling results.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
In Elasticsearch, via nested and child documents, we can have complex data models. Elasticsearch, by default, returns only documents that match the searched type and not the nested/children one that matches the query.
The inner_hits function is introduced in Elasticsearch 5.x to provide this functionality.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
It's very common for users to commit typing errors or to require suggestions for words that they are writing. These issues are solved by Elasticsearch with the suggest functionality.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
It is often required to return only the count of the matched results and not the results themselves.
There are a lot of scenarios involving counting, such as the following:
To return the number of something (how many posts for a blog, how many comments for a post)
Validating whether some items are available. Are there posts? Are there comments?
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
When executing searches, it's very common to have documents that don't match the query as expected. To easily debug these scenarios, Elasticsearch provides the explain query call.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
A new feature available in Elasticsearch 5.x is the profile API. This allows the user to track the time spent by Elasticsearch in executing a search or an aggregation.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
We saw how to delete a document in the recipe Deleting a document in Chapter 4, Basic Operations. Deleting a document is very fast but it requires knowing the document ID.
Elasticsearch provides a call to delete all the documents that match a query via an additional module called reindex, which is installed by default.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command-line you need to install curl for your operating system.
To correctly execute the following commands you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
In the previous chapter, we saw how to update a document (Chapter 4, Basic Operations, Update a document recipe).
The update_by_query API call allows the user to execute the update on all the documents that match a query. It is very useful if you need to do the following:
Reindex a subset of your records that match a query. It's common if you change your document mapping and need the documents to be reprocessed.
Update values of your records the match a query.
This functionality is provided by an additional module called reindex that is installed by default.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
One of the most common queries, usually in conjunction with a filter, is the match_all query. This kind of query allows the user to return all the documents. It's often used in conjunction with filters.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need to install curl for your operating system.
To correctly execute the following commands, you will need an index populated with the chapter_05/populate_query.sh script available in the online code.
In order to execute a match_all query, we will perform the following steps:
From the command line, we execute the query, as follows:
curl -XPOST 'http://127.0.0.1:9200/test-index/test-
type/_search?pretty' -d '{"query": {"match_all" : {}}}'
The result returned by Elasticsearch, if everything works, should be as follows:
...
Most people using a search engine have at sometime used the syntax with minus (-) and plus (+) to include or exclude query terms. The Boolean query allows the user to programmatically define queries to include, exclude, optionally include (should), or filter in the query.
This kind of query is one of the most important ones because it allows the user to aggregate a lot of simple queries/filters that we will see in this chapter to build a big complex one.
Two main concepts are important in searches: query and filter. The query means that the matched results are scored using an internal Lucene scoring algorithm; for the filter, the results are matched without scoring. Because the filter doesn't need to compute the score, it is generally faster and can be cached.
You will need an up-and-running Elasticsearch installation as used in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via a command line, you need...
© 2017 Packt Publishing Limited All Rights Reserved
