Work in progress |
The Trend Detection analysis can be used to detect unusual trends in the time series generated by the number-of-items per time-unit in a particular project for a particular query. The detection of unusual trends is done by learning from the historical/old data to auto-compute a reasonable threshold. So, in order for it to work properly it is important that we have historical data.
As an example scenario, consider a project `News` with a feed of all the news-items from a few of the major news publications. Now, a query like `Facebook AND Whatsapp` will filter the list of all the documents to a sub-list of documents/news-items containing both the words Facebook and Whatsapp. For the project `News` with query `Facebook AND Whatsapp`, we define a time-series as the number of items matching `Facebook AND Whatsapp` per time-unit, where time-unit can be hourly, weekly, daily, monthly or yearly.
The Trend Detection service can now be used with the Squirro client. In order to set up a new trend-detection on an existing project, one can use the `new_trenddetection` method of the squirro client to set up the trend detection for a particular query `q` on a project with project-id `project-id`. A valid email-address is required for generating an email-alert when something unusual is detected for a particular query-projectid pair. Parameter `aggregation_interval` controls the discretization of the time series (`time-unit` parameter above) before initiating the trend-detection analysis. An example usage can beseen below.
>>> client.new_trenddetection( project_id='2sic33jZTi-ifflvQAVcfw', query='hello world', name='test name', email_user='test@squirro.com', aggregation_interval='1w') |
Once, a new trend-detection analysis has been created, one will receive alert emails everytime the number of items in a particular time window (defined by the time-unit) goes above the automatically computed threshold for that period. Moreover, one can also see the trends detected in the historical data using the `get_trenddetection_labels` method of the squirro client. No alert email will be generated for these trends detected in the historical data.
>>> client.get_trenddetection_thresholds(project_id='2sic33jZTi-ifflvQAVcfw', trenddetection_id='fd5x9NIqQbyBmF4Yph9MMw') >>> client.get_trenddetection_labels(project_id='2sic33jZTi-ifflvQAVcfw', trenddetection_id='fd5x9NIqQbyBmF4Yph9MMw') |