Chapter 8. What Next?

So far we have learned pretty much everything about Sphinx and its usage. In this chapter we will explore a few of the newer features that were introduced in recent versions of Sphinx.

We will also see what scenarios Sphinx is useful in, and list a few popular websites that use Sphinx to power their search engines.

SphinxQL

Programmers normally issue search queries using one or more client libraries that relate to the database on which the search is to be performed. Some programmers may also find it easier to write an SQL query than to use the Sphinx Client API library.

SphinxQL is used to issue search queries in the form of SQL queries. These queries can be fired from any client of the database in question, and returns the results in the way that a normal query would. Currently MySQL binary network protocol is supported and this enables Sphinx to be accessed with the regular MySQL API.

SphinxQL in action

Let's take an example of MySQL CLI client program and see how we can use it to query Sphinx. We will use the same database and configuration file that we created in Chapter 7, Sphinx Configuration (for distributed searching).

Note

The following exercise uses the items database table from Chapter 7,Sphinx Configuration.

Time for action - querying Sphinx using MySQL CLI

1. 1. Create the file /usr/local/sphinx/etc/sphinx-ql.conf and add the following code:

   source items
   {
   type = mysql
   sql_host = localhost
   sql_user = root
   sql_pass =
   sql_db = sphinx_conf
   sql_query = SELECT id, title, content, \
   UNIX_TIMESTAMP(created) AS created FROM items
   sql_attr_timestamp = created
   }
   index items
   {
   source = items
   path = /usr/local/sphinx/var/data/items-ql
   charset_type = utf-8
   }
   searchd
   {
   listen = localhost:9306:mysql41
   log = /usr/local/sphinx/var/log/ql-searchd.log
   query_log = /usr/local/sphinx/var/log/ql-query.log
   pid_file = /usr/local/sphinx/var/log/ql-searchd.pid
   }
   

2. 2. Create the index by running the indexer utility:

   $/usr/local/sphinx/bin/indexer -c /usr/local/sphinx/etc/sphinx-ql.conf --all
   

3. 3. Start the searchd daemon:

   $/usr/local/sphinx/bin/searchd -c /usr/local/sphinx/etc/sphinx-ql.conf
   

4. 4. Connect to the MySQL CLI program:

   $mysql -u dbuser pdbpass -h localhost -P 9306
   

5. 5. The previous command will connect to the MySQL server...

Use case scenarios

Sphinx can be used in any web application that involves the searching of data. It can be a simple blog, shopping portal, or a very complex hotel or airline booking website. The most common use case of Sphinx is where searching needs to be fast and reliable.

The following is a list of scenarios where Sphinx will perform best:

• Applications with a lot (billions of documents) of content

• Applications that need to filter their search results based on numerous attributes (which are not full-text)

• Applications where search is required on segregated data

• Applications where data is coming from a non-conventional source such as a file system, mailboxes, NoSQL databases, and so on

• Applications that need distributed searching so that load is balanced

Popular websites using Sphinx

Sphinx is gaining popularity day by day and many high traffic websites are migrating their search engines to Sphinx. A few who have already done this are listed next (Ref: http://sphinxsearch.com/info/powered/):

• http://craigslist.org/: Craigslist is one of the world's most popular and highly visited websites. Thousands of people search the classifieds on craigslist every hour. They moved to using Sphinx in November 2008. Craigslist is known to have the busiest Sphinx server in the world.

• http://www.joomla.org/: Sphinx is used to improve search through their forum, which comprises more than 700,000 posts.

• http://mininova.org/: One of the biggest sites to search for torrents. Sphinx on this site servers more than five million searches per day.

• http://boardreader.com/: The website with the biggest Sphinx installation in terms of data. It is indexing over two billion documents now.

• http://netlog.com/: This is a huge social networking site with over 35 million...

Summary

In this chapter:

• We learned about SphinxQL, which can be accessed using a regular MySQL client

• We saw different statements supported by SphinxQL

• We saw different cases when Sphinx can be used

• We saw a few popular websites (and some statistics) that use Sphinx

We have covered all the topics that should get you started with using Sphinx in your projects. The beauty of Sphinx is that it is almost language independent. Sphinx client API implementations are available for numerous programming languages.