This is the first minor release as the follow up to the last major Squirro release 3.2.0. This release was released on Nov 27, 2020.

New Features

• Support for document level tagging in DSS. This means that we now support document level tagging as well as ML templates tuned for document level tagging as well. This comes in addition to the sentence level support that was released in the first iteration of DSS in release 3.2.0

• Support for Single Logout in the extauth_saml plugin. With the latest studio plugin & this Squirro release, it can be tried out on any Squirro instance connected with Squirro ID.

Open

• We now finally allow import of Squirro projects exported on Squirro Version 3.1.0 or later on the latest Squirro Release. We intend to keep this compatibility going forward for future releases also.

Open

Improvements

• Make the timeout for plumber service configurable to allow for pipelets to take more time for their processing without running into timeout issues. This setting can be configured with parameter step_plumber_timeout_secs in the pipeline section of common.ini.

• Improved ingester logging for backoffs during ingestion of batches with errors.

• Update version of PySaml2 to the latest version 6.3.1.

• Support for reading relevant config from /etc/squirro to establish connection to ES & Squirro services. This allows for the tool to work correctly even when you have setup SSL for Squirro services as well as ES.

• More powerful squirro_stop, squirro_start, and squirro_restart commands. Now you can simply say sudo squirro_start (or similar) in situation where your user has the permissions to run the systemctl commands as sudo and Squirro will do the right thing and use sudo to start/stop/restart all relevant squirro services. Furthermore, we skip cluster service from these commands to avoid having accidental failovers (even if the /etc/squirro/cluster.ini) file exists.

• Python3 fixes around KEE.

• Migration scripts for topic service now run at the boot time of topic service and are no longer run during the upgrade time. This allows us to make sure that our databases are in a consistant state every time the service boots up. If this becomes a problem for you, this migrations at the boot time of the service can be disabled by setting the execute_at_startup in migration section of the relevant service file as False.

• Optimise Item fetching for Machine Learning workflows. squirro_query_loader in lib/nlp only fetches the desired number of documents when the default batch_size is larger than the requested documents.

• If dataloader is run with the delete flag, we do not delete any items which have missing IDs.

• squirro_source plugin now uses the inbuilt key_value_store and key_value_cache APIs for its state management and duplication detection logic.

• KEE upload also uploads the tokenizers if they exist.

• Added German translation strings for DSS interface.

• Rules defined during DSS labelling now also stores the corresponding label and hence can be visualised in the rules overview page.

• Squirro Frontend SDK now allows passing a pre-defined query as a URL parameter.

• Additional Highcharts modules are loaded by default, allowing the building of custom gauge widgets

• Ingestion pipeline using a DSS model now has an edit button next to the DSS model step for a quick review of key model metrics.

• DSS has a new Beta logo.

Fixes

• Fixed a bug where migrating to https based communication between Squirro micro-services would result in broken HTTP cache invalidation logic.

• Fixed a bug where internal Squirro Users (used for dataloading) were leaking into the Frontend interface.

• https://squirro.atlassian.net/browse/SQ-12242

Breaking Changes

• squirro_activate command now activates the python3 virtualenv as opposed to python2 virtualenv. If the python2 virtualenv is still present, it will print a warning with a message on how to activate the python2 virtualenv manually.

Fresh Installation Instructions

Please follow the regular installation steps

Added on December 4, 2020 (build X - Patch 1)

• Fixed default dashboard store being ignored in shared dashboards and SDK

Added on December 10, 2020 (build X - Patch 2)

• Added support for select() in Highcharts widget Custom Widget API

• Exposed Highcharts Sunburst chart to Custom Widget API

• Fixed update flow of the Highcharts widget

• Fixed an issue which prevented the Highcharts widget from rendering if it was on a hidden layer during initial render of the dashboard

Upgrade Instructions