Chapter 3. Designing Schemas

Now that we have seen how to install Solr on our machine, let's dive deeper and understand the nitty-gritty of Solr.

Assume that you are building a home that you have always desired. How would you start? Will you just get all the bricks, cement, windows, doors, beams, and so on and ask the builders to start building? Nope! You would want to make sure you go through various designs based on the area that you have and decide on a design that you think will not only look good but also last long. Creating any application follows the same principle and demands proper schema design.

In this chapter, we will traverse through schema design. We will understand how to design a schema using documents and fields. We will also see various field types and get an understanding of the Schema API. We will finally look at schemaless mode.

How Solr works

The easiest way to understand how Solr works is to see how a telephone directory helps you to look something up. A telephone directory, or yellow pages as it is called in some places, is a book containing lots of phone numbers. It has lots of pages. Now, to find information in it would be a humongous task unless it had some sort of indexing and categorizing. For example, we can easily find all the restaurants by just navigating to the category of restaurants and finding the locality that we are living nearby.

Similarly, Solr can be imagined as a huge directory that has been fed data as per our requirement, and it can be queried to get the relevant data by using an appropriate search criteria that was indexed while feeding in the data. Let's have a look at the following diagram and understand how Solr search platform works:

As you can see, the way to look at Solr is like this—it is basically fed with lots of information, which is correctly indexed. Then, in order to retrieve...

Understanding field types

As discussed earlier, we are able to tell Solr how it should interpret the incoming data in a field and how we can query a field using the information specified in field types.

Definitions and properties of field types

Before going to the definitions and properties, we will see what field analysis means.

What Solr should do or how it should interpret data whenever data is indexed is important. For example, a description of a book can contain lots of useless words: helping verbs such as is, was, and are; pronouns such as they, we, and so on; and other general words such as the, a, this, and so on. Querying these words will bring all the data. Similarly what should we do with words that have capital letters?

All of these problems can be catered using field analysis to ignore common words or casing while indexing or querying. We will dive deep into field analysis in the next chapter.

Now, coming back to field types, all analyses on a field are done by the field type, whether...

Field management

Once your primary work of field types setup is done, field definition is a small task. Just as with field types, the fields element of schema.xml holds the field definition.

Field properties

Let's first see a sample field definition:

<field name="weight" type="float" default=”0.0” indexed="true" stored="true"/>

In the preceding example, we have defined a field named weight, whose field type is float with a default value of 0.0. Moreover, the indexed as well as stored properties are explicitly set to true.

Field definitions will have these properties:

• name: The field name. This has to be alphanumeric and can include underscore characters. It cannot begin with a digit. Reserved names should start and end with underscores (for example, _root_). Every field must have a name.
• type: The name of the fieldType. All the fields should have a type.
• default: The default value to be used for the field.

Fields and field types share many of the optional properties here. If there are two different...

Mastering Schema API

Schema API is the one-stop shop for most operations on your schema. It provides a REST-like HTTP API for doing all these operations.

You can read, write, or delete dynamic fields, fields, copy field rules, and field types. 

Note

Do not manually write any changes into the managed-schema file yourself. This will work only as long as you don't use Schema API. If you use Schema API by mistake, all your changes might be overwritten. So, it is highly recommend that you leave your managed-schema file alone.

The response of the API call is of either JSON or XML format. 

Assuming that you are using the gettingstarted collection, the base address of API will be http://localhost:8983/solr/gettingstarted.

Note

Always reindex once you use Schema API for modifications. Only then will the changes that you have applied to the schema be reflected for existing documents that are already indexed.

Schema API in detail

Let's see some of the important schema endpoints. We will do all the examples...

Deciphering schemaless mode

Schemaless mode is used when we want to quickly create a useful schema by indexing sample data. It does not involve any manual editing of the data.

All of its features are managed by solrconfig.xml.

The features that we are particularly interested in are:

• Managed schema: All modifications in the schema are made via Solr API at runtime using schemaFactory, which supports these changes.
• Field value class guessing: This is a technique of using a cascading set of parsers on fields that have not been seen before. It then guesses whether the field is an Integer, Long, Float, Double, Boolean, or Date.

And finally used for automatic schema field addition that is based on field value classes.

Creating a schemaless example

All of the preceding three features are already configured in the Solr bundle. To start using schemaless mode, run the following command:

bin/solr start -e schemaless

This will start a single Solr server with the collection gettingstarted.

In order to see the schema...

Summary

In this chapter, we got an overview of how Solr works and saw schema design. We then jumped into Solr field types and saw how to define fields, copy fields, and create dynamic fields. We moved on to the Schema API, and finally we saw what schemaless mode is all about.

In the next chapter, we will get our hands dirty and learn all about analyzers, tokenizers, and filters.