Dealing with a huge count of open files

In this section, we will learn how to get rid of exceptions thrown due to a huge number of files that are open. Before you get into this section, it is recommended that you refer to one of the preceding sections called Reducing the file count in the index.

1. For the purpose of demonstration, let us assume that Solr (running on a Unix environment) throws the exception whose header looks as follows:

   java.io.FileNotFoundException: /use/share/solr/data/index/_8.tii
   

   This shows that there are too many open files.

2. We will increase the opened files' limit from 1000 (this was earlier set in my case, and is prone to differ) to 3000. To do so, we will use the ulimit command-line utility as follows:

   ulimit –n 3000
   

3. Stopping at this stage would just prove to be a workaround. The primary cause behind this exception is the huge number of segment files that constitute an index. So, the immediate activity proceeding with the ulimit utility should be to optimize the index...