Work in progress
What is Trend Detection?
The Trend Detection service can now be used with the Squirro client. The Trend Detection analysis can be used to detect unusual trends in the time series generated by the number-of-items per time-unit in a particular project for a particular query.
As an example scenario, consider a project `News` with a feed of all the news-items from a few of the major news publications. Now, a query like `Facebook AND Whatsapp` will filter the list of all the documents to a sub-list of documents/news-items containing both the words Facebook and Whatsapp. For the project `News` with query `Facebook AND Whatsapp`, we define a time-series as the number of items matching `Facebook AND Whatsapp` per time-unit, where time-unit can be hourly, weekly, daily, monthly and yearly.
How to set up Trend Detection?
In order to set up a new trend-detection on an existing project, one can use the `new_trenddetection` method of the squirro client to set up the trend detection for a particular query `q` on a project with project-id `project-id`. A valid email-address is required for generating an email-alert when something unusual is detected for a particular query-projectid pair. Parameter `aggregation_interval` controls the discretization of the time series (`time-unit` parameter above) before initiating the trend-detection analysis. An example usage can beseen below.
>>> client.new_trenddetection(
project_id='2sic33jZTi-ifflvQAVcfw',
query='hello world',
name='test name',
email_user='test@squirro.com',
aggregation_interval='1w')
Since,
How to inspect detected Trends?
Once, a new trend-detection analysis has been created, one will receive alert emails everytime the number of items in a particular time window (defined by the time-unit) goes above the automatically computed threshold for that period. Moreover, one can also see the trends detected in the historical data using the `get_trenddetection_labels` method of the squirro client. No alert email will be generated for these trends detected in the historical data.
>>> client.get_trenddetection_thresholds(project_id='2sic33jZTi-ifflvQAVcfw', trenddetection_id='fd5x9NIqQbyBmF4Yph9MMw') >>> client.get_trenddetection_labels(project_id='2sic33jZTi-ifflvQAVcfw', trenddetection_id='fd5x9NIqQbyBmF4Yph9MMw')