Chapter 7. Spell Check in Solr

The spell check component can be used to suggest spelling corrections based on the data we have in our index. In this chapter, we will see how to enable spell check in our index and use PHP to get and display spelling corrections. The topics that we will cover in this chapter are as follows:

• Solr configuration for spell check

• Spell checker implementations available in Solr

• Running a spell check query using PHP

• Displaying suggestions and collations

• Building the autocomplete feature

Note

Spell check works on indexed words. If our index has incorrect spellings, the suggestions may also be misspelled.

Spell check can be used to suggest spelling corrections to the user by providing a did you mean functionality. It is similar to the showing results for feature that Google provides. It can be used to provide a list of suggestions for autocompleting the user's input text. PHP also has a similar functionality known as pspell but this spellcheck is built on top of the index...

Solr configuration for spell check

The demo schema and configuration that comes with Solr installation already has spell check configured in it. Let us look at its settings:

1. Open up solrconfig.xml inside <solr_dir>/example/solr/collection1/conf.

2. Search for searchComponent by the name of spellcheck.

3. Inside the spellcheck component there are multiple spellchecker(s). Here is the default spellchecker that comes along with Solr:

   <lst name="spellchecker">
   <str name="name">default</str>
   <str name="field">text</str>
   <str name="classname">solr.DirectSolrSpellChecker</str>
   <float name="accuracy">0.5</float>
   <int name="maxEdits">2</int>
   <int name="minPrefix">1</int>
   <int name="maxInspections">5</int>
   <int name="minQueryLength">4</int>
   <float name="maxQueryFrequency">0.01</float>
   <float name="thresholdTokenFrequency">.01</float>
   </lst>

4. The preceding block of code shows...

Spell checker implementations available with Solr

Let us go through the different spell checker implementations available with Solr:

• DirectSolrSpellChecker: This implementation does not require a separate index to be built for spell checking. It uses the main Solr index for spelling suggestions.

• IndexBasedSpellChecker: This implementation is used to create and maintain a spelling dictionary that is based on the Solr index. Since a separate index is created and maintained, we need to build/rebuild the index whenever the main index changes. This can be done automatically by enabling buildOnCommit or buildOnOptimize in the configuration. Also, we need to specify the location of the index to be created using the spellcheckIndexDir variable in our Solr spellcheck component configuration.

  Note

  The buildOnCommit component is very expensive. It is recommended to use buildOnOptimize or explicit build using spellcheck.build=true in Solr URL.

• FileBasedSpellChecker: This implementation uses a flat file...

Running a spell check query using PHP

Let us configure Solr so that spell check happens on two fields, name and author:

1. Change the contents of the schema.xml file. Create a new field on which spellcheck will happen and copy the name and author fields to the new field using the following code:

   <field name="spellfld" type="text_general" indexed="true" stored="false" multiValued="true"/>
   <copyField source="name" dest="spellfld"/>
   <copyField source="author" dest="spellfld"/>

2. Change the spellchecker field for the default spellchecker in solrconfig.xml to the new field we have just created. The default spellchecker uses the DirectSolrSpellChecker implementation of spell checker available with Solr.

   <lst name="spellchecker">
   <str name="name">default</str>
   <str name="field">spellfld</str>

3. By default the /select request handler in Solr configuration does not have spellcheck settings and results. So let us add these variables in requestHandler named /select...

Implementing the autocomplete feature using PHP and Solr

Autocomplete feature can be built by creating a Suggester in Solr and using the Suggester available in Solarium. The purpose of autocomplete is to suggest query terms based on incomplete user input. Suggester works very similarly to the spellcheck feature. It can be made to work either on the main index or any other dictionary.

First let us change the schema.xml file to add a spellcheck component named suggest:

<searchComponent name="suggest" class="solr.SpellCheckComponent">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="field">suggestfld</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookup</str>
<str name="storeDir">suggest_idx</str>
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
</lst>
<...

Summary

We started with an understanding of how spell check works on Solr. We went through the configuration of Solr for creating spell check index and saw different implementations of the spellchecker available with Solr. We understood some of the fine tuning options available in spell check in Solr. Next, we created a field in Solr for doing spell check on book name and author and configured Solr to provide spelling suggestions using this field. We saw a variation of spell check that can be used to provide spelling suggestions for autocompletion. We created a separate Solr index for autocomplete suggestions and saw PHP code that takes a three-character word and provides suggestions from the index.