Page Comparison

Sorting by time and relevance

The results of a given query are sorted by relevance score. Relevance score is computed based on some factors like term frequency, document frequency, where the querying terms matched in the document (body, title, summary...). The limitation of sorting purely by relevance is it ignores the important of up-to-date content such as news articles.

Using the combination of time and relevance in your project definition, Squirro will also sort the result list by using the created_at field of each item i.e. the most recent documents are put first in the result list. The limitation of of sorting purely by time field is that the most recent documents are often poor matches for the original query.

How Squirro calculates time and relevance

To combine time and relevance to the ranking, we use a score we call time_relevance which combines both time and relevance factors into the final score using the following formula:

...

• relevance_score: original relevance score of document wrt the query
• base: parameter use to adjust the impact of relevance. Higher this value, higher impact of relevance score (if you set this value very high, e.g 100, you can see impact of time factor is very minor, relevance score contributes mostly in final score)
• range: range of decay, higher this value, longer decay range (if range = 0 you will see final score depends only on relevance score)
• decay: decay speed, higher this value, quicker decay value (if decay = 0 you will see final score depends only on relevance score)
• age_days: number of days from now since created_at time of documents.

...

Beside above parameters we introduce as well low_relevance and old_period parameters to handle edge cases where very new document with very low relevance or very old document with very high relevance should be always put at the end of result list. Default value is 180 days for old_period and 0.25 for low_relevance; looking at above curve of relevance scores you can see the relevance scores are quite flat after value 0.25)

How to Configure

Squirro search results can be sorted in by time_relevance as default on a per-project basis by setting the Advanced Options.

The factors of ranking formula (base, range, decay, low_relevance, old_period) can be tuned in in common.ini, in the [ranking] section

But why use not Elasticsearch's decay function?

Expert Elasticsearch users may be familiar with it's decay function which scores a document with a function that decays depending on the distance of a numeric field value of the document from a user given origin. This function is useful for some use cases like finding hotel which close to a geo point, finding a restaurant with a cheap price. However is not easy to apply this function to Squirro results for two main reasons;

...