During indexing, whenever a document is deleted or updated, the document is marked as deleted in its original segment. This generates some percentage of waste storage because the index will contain around 15% to 20% of deleted documents. Merging the Lucene Segment process will control this ratio with time, in order to maintain it as low as possible. However, in some situations, especially after a bulk ingestion, the percentage of deleted documents can be up to 50%. This percentage is determined by the ratio of numDocs to maxDocs which can be set in the Solr Admin interface.
Since optimizing the index is not a recommended operation in many use cases, this option will remove the deleted documents from your index. However, it will create segments which are much larger than the maximum considered for future merges. If you are optimizing your index periodically and can afford the time to optimize every time you rebuild your index, then optimizing is reasonable and it will increase the searching performance.
This operation can be performed using the SOLR REST API by default available it is available at http://127.0.0.1:8983/solr/alfresco/update?optimize=true or by clicking the Optimize now button in the Core > Overview section of the Solr Admin interface.
You can optimize the index by reducing it to N segments with N >= 1.
http://127.0.0.1:8983/solr/alfresco/update?optimize=true&maxSegments=N
This can be useful for reducing the impact of the force merge operation. The advantages of using N >= 1 are:
- The force merge execution takes less resources.
- Avoids the production of a single large segment.
The value of N must be chosen carefully. N should be smaller than the current number of segments. Moreover, it is possible that some segments are not selected for merging. Consequentially, not all the deleted documents maybe removed from the index.