Fulltext Search extremly slow with large result sets (>1k docs)

[Matthias Bauer <matthias.bauer.drs@…>]

Fulltext search was extremely slow with large result sets. Example: The search for a quite common search term returns some 12k documents. This search took 45 secs for initial query and 15 secs for successive queries. Executing the same query on the same index data using lucene's index analysis tool luke (​https://code.google.com/p/luke/) just takes some 20 ms.

Profiling showed that retrieving the dataset in DocumentHitCollector?.java:71 is causing this slow down. At this place, we were retrieving the complete data set including the full text-extract ("content" field). But in this part of the code we only need the DocID, BranchID and LanguageID fields. The fix (see attached patch) instructs lucene to only retrieve the fields we need from the index.

This improves the search time for the example a lot. We get 17 secs for the initial search and 280 ms for successive searches (when everything is cached).

[paul]

In [5912]:

fixes #841. Apply patch by Matthias Bauer for FTS speedup. Thanks Matthias

[anonymous]

Exercise is a good way to keep leg cramps at bay.But I will give you an antibiotic prescription just in case, and if the fever keeps up for another four or five days, you can start the antibiotic.Some people aren't aware that excess carbohydrates can easily cause weight gain.Mammography is known for its controversial ill effects and inaccuracies and research has shown that it is difficult to detect tumors since the breast tissue is denser among younger women. ​pregnancy diet for overweight women To decrease dampness and heat According to traditional Chinese medicine, Overweight American can be caused by typical American diet which contributes to dampness accumulation in the body's organ as a result of kidney abnormal function in fluid distribution and transportation, leading to dampness descenting to abnormal region that distorts the normal conception.If you are not sure why you want to lose weight, then you must write the reasons why you want to lose weight.Avoid sleeping very late at night.Of course, it isn't too hard to believe that the tabloid would want to jump on the huge news story.