We're excited to announce Squirro 3.1.0, released on 20th of July 2020.

Contents - What's in the release?

New Features

Widget Editor

It is now possible to permanently associate widgets with GitHub repositories for a deeper level of integration
The GitHub flow has been reworked, allowing quickly detecting changes between the widget and the repo

Layout Master in Squirro Dashboards

We have added a new concept "Layout Master" to this release. Please refer to Layout Master Dashboard for more details.

Added option to replicate the precise master section setup in any dashboard through the Sync functionality
Added option to align dashboard tabs with the center section

Synonyms

Starting with Squirro 3.1.0 release, we have made the support for Synonyms for your search requests even more powerful. While it has already been possible to use synonyms with specific facets using our APIs for a few years, we have also made it possible now to use them on the Title, Body as well as the Abstract of your Squirro Items. There is more, you can do all of that now finally over the Frontend with just a few clicks. If you are interested in knowing more about it, please read our Using Synonyms documentation page.

Spellcheck

Squirro 3.1.0 exposes the ability to spellcheck user queries based on the stored data in Squirro.
For further information please see the documentation page Spellchecking BE.

Significant Terms Extractor

Given a set of documents, the goal of significant terms extractor is to identify in the text the set of words that can potentially describe a document. It is an unsupervised technique that constantly adapts itself at every call to maintain the model always up to date.

For further information please refer to Significant Terms Extraction in Squirro

ElasticSearch 7.7

There has been a major migration of the Elasticsearch engine that is supported by Squirro. We have moved from using the version 6.2.2 to 7.7.0.

We were careful not to introduce new features per-se during the migration. It did, however, facilitate the deployment of extra functionality to our platform like encrypted connectivity with ES (see below)

Security Improvements (ES, MariaDB)

With Squirro 3.1.0 you have the possibility to encrypt Squirro communication with MariaDB and Elasticsearch.

For further information please refer to Securing Elasticsearch and MariaDB over TLS

Improved Entities Support

Starting with Squirro 3.1.0 release, Squirro entity data-model has become even more powerful. It now allows saving for more than one value in a specific entity property. What does all of that even mean . It means that now you can finally add multiple companies to a single entity. In addition to that, we have also made it possible to add more than one entities to a single sentence. This will help you catch all those sentences which do not quite fit one single concept in your business.

Please note that this is a breaking change. If you have code which directly creates entities using our APIs, please contact Squirro Support.

Layer arrangement (composite layers)

Allowed to change the default widget arrangement on a layer
Three widget arrangements available: Grid (default for Squirro 3.0 and below), Accordion and Carousel
Accordion arrangement packs all widgets on a layer into an accordion style control
- Allows for creation of menu-style navigation as found on Squirro Explore page
Carousel arrangement packs all widgets on a layer into a carousel style control
- Allows displaying multiple widgets in the same visual space without the need for a layer-based implementation

Profile Avatar

Users can now customise their avatar (profile picture)
Exposed the user avatar into Custom Widget API

Squirro Dashboard SDK

The SDK now parses well under strict mode, allowing for use in environments that require it (like Salesforce API v40+)
Optimised several techniques used in the Dashboard component, allowing usage under Salesforce Lightning

Restrict Dashboard width

For better UX, config. option under app n navbar settings to select max. width for dashboard resize, beyond which white space grows on either side.
Options: 1024, 1366, 1440 and 1680 to select from so that they make conscious decisions wrt. the devices they use.

Improvements

Sentence Splitting improvement
Card shadows, font sizes, border radius improvements
Table widget now configurable by multiple facets
Bigger searchbar, tabs, buttons, etc. as 1 unit height restriction on certain widgets removed
Configurable tooltips for the icons displayed inside link widget and facets widget
Facet dropdown multi-select: The buttons are always visible now.

Bug Fixes

SB-136 - Rich text formatting issues
SB-313 - Can't type 's' on RTE widget when having an item detail widget
SB-349 - Small language change for email validation
SB-370 - Support array on userPref
SB-395 - RTE widget content is not scrollable
SB-396 - Breadcrumb widget action chips always visible
SB-413 - Users page number of results dropdown doesn't always work
SB-414 - Request demo url is wrong
SB-415 - Catalyst mapping in incorrect classification feedback popup is not working correctly!
SB-416 - Selection tagging is removing consecutive spaces from sentence
SQ-10748 - Force sending empty array when all tags are removed from the single facet
SQ-10751 - Cards Widget: Padding on CTA is wrong
SQ-10880 - Entities highlights not working on result widget with only entities enabled
SQ-10886 - implement-multi-value-selection-for-facets-list
SQ-11172 - Replacing long URLS with only the domain name or source name in case of PDF
SQ-11219 - Users List filter dropdown doesn't always work
SQ-11261 - Facets List Widget: Facet List Order config does not work
SQ-11265 - Tag Cloud widget: Legend is always empty
SQ-11266 - Trend Widget: Y axis label is in the wrong side
SQ-11277 - Direct access to search tagging edit url fails
SQ-11283 - Sort loaders alphabetically
SQ-11311 - Line height for share item is wrong
SQ-11319 - Link widget delete icon issues
SQ-11370 - Do not limit Link widget to 4 columns
SQ-11376 - Smartfilter widget does not render in edit mode
SQ-11410 - Topic cluster filtering is not working
SQ-11571 - Prevent fetching recommendation catalysts on item detail if not needed
Only allow master dashboard promotion and demotion for admins

Added on July 24, 2020 (build 36 - Patch 1)

Fix loading of themes

Added on August 11, 2020 (build 38 - Patch 2)

Added multi-value properties for entities and multiple entities per sentence.
Adjusted entity UI to allow User or Admin mode for displaying of entity popups.

Added on October 8, 2020 (build 41 - Patch 4)

Increased file uploader timeout from 30 seconds to 10 minutes.

Added on October 12, 2020 (build 42 - Patch 5)

SB-487- Fix broken search after a project reset in a project with synonyms
SB-467 - Error re-running the pipelets with entities
SSO & SAML improvements
SB-472 Terms extractor fixes
- Python 3 compatibility
- Performance improvements
- NLTK language models are downloaded if not available
Add support for german language

Added on October 15, 2020 (build 43 - Patch 6)

SB-484 - Fixed wrong highlighting for PDF (duplicated words)
Exposed base Collections and Models globals to custom widgets
Prevent against code breakage due to scenarios where a project has None synonym_Config

Added on October 19, 2020 (build 44 - Patch 7)

Added and documented new Widget API method, getCustomEvents()

Added on November 5, 2020 (build 45 - Patch 8)

Facets Bar Chart: Do not cache models to allow collection tweaking

Added on November 6, 2020 (build 46 - Patch 9)

Fix aggregation collections caching in Pie Chart

Fresh Installation Instructions

Please follow the regular installation steps

Upgrade Instructions

Please ensure that your current version is 3.0.0 If you are on a version older than 3.0.0, please contact support.

With this & every subsequent upgrade, all services will run in python3.6.

If you have not migrated & tested your custom dataloader plugins, studio plugins or piplets to python3.6 compatible code, you can still manually revert to run specific services in python2.7 by following the steps outlined here.

This is the last release which continues python 2.7 support. Therefore, we strongly recommend that you migrate any custom code now and proceed with the new default python 3.6.

# stop squirro
squirro_stop

yum install java-1.8.0-openjdk
yum update java-1.8.0-openjdk
yum update squirro-storage-node-users

# Ensure that `path.data` variable is set in ES. If you have used Ansible in the past to deploy Squirro, you will be affected by this bug.
grep -q "^path.data" /etc/elasticsearch/elasticsearch.yml ||  echo "path.data: /usr/share/elasticsearch/data/" >> /etc/elasticsearch/elasticsearch.yml

# update elasticsearch
# Make sure that you delete any indices which have been created with squirro_v8 template. 
# curl http://localhost:9200/_cat/indices -> then inspect for occurrences of squirro_v8 in the output.
# If your cluster has been created with Squirro 2.6.0, released on April 3rd 2018 or later, 
# you do not have to worry about this.
# To delete: curl -XDELETE http://localhost:9200/squirro_v8*
# To delete old jaeger indices: curl -XDELETE http://localhost:9200/jaeger*

systemctl stop elasticsearch
yum update elasticsearch

# Disregard the ES rpmnew warnings as they will be resolved by `squirro-storage-node` cluster

# update storage node
yum update squirro-storage-node
# wait for elasticsearch status to become green, only then
# update cluster node
yum update squirro-cluster-node-users
yum update squirro-*
squirro_start

Resolve all the `.rpmnew` files in `/etc/squirro`. This process involves merging the changes between the `.ini` (e.g., `storage.ini`) and '.ini.rpmnew`(e.g., `storage.ini.rpmnew`) files and then eventually deleting the `.ini.rpmnew` files. Finally, restart the services with merged `.ini` files (or just use `squirro_restart` to restart all services).

Stop squirro on the cluster node

squirro_stop

Upgrade the storage node by running:

yum install java-1.8.0-openjdk
yum update java-1.8.0-openjdk
yum update squirro-storage-node-users

# Ensure that `path.data` variable is set in ES. If you have used Ansible in the past to deploy Squirro, you will be affected by this bug.
grep -q "^path.data" /etc/elasticsearch/elasticsearch.yml ||  echo "path.data: /usr/share/elasticsearch/data/" >> /etc/elasticsearch/elasticsearch.yml

# update elasticsearch
# Make sure that you delete any indices which have been created with squirro_v8 template. 
# curl http://localhost:9200/_cat/indices -> then inspect for occurrences of squirro_v8 in the output.
# If your cluster has been created with Squirro 2.6.0, released on April 3rd 2018 or later, 
# you do not have to worry about this.
# To delete: curl -XDELETE http://localhost:9200/squirro_v8*
# To delete old jaeger indices: curl -XDELETE http://localhost:9200/jaeger*

systemctl stop elasticsearch
yum update elasticsearch

# Disregard the ES rpmnew warnings as they will be resolved by `squirro-storage-node` cluster


#update storage node
yum update squirro-storage-node

# wait for elasticsearch status to become green, only then update the clusternode

Upgrade the cluster node by running:

yum install java-1.8.0-openjdk
yum update java-1.8.0-openjdk
yum update squirro-cluster-node-users
yum update squirro-*
squirro_start

Resolve all the `.rpmnew` files in `/etc/squirro`. This process involves merging the changes between the `.ini` (e.g., `storage.ini`) and '.ini.rpmnew`(e.g., `storage.ini.rpmnew`) files and then eventually deleting the `.ini.rpmnew` files. Finally, restart the services with merged `.ini` files (or just use `squirro_restart` to restart all services).

Stop squirro on all cluster nodes

squirro_stop

Stop elasticsearch on all storage nodes

# Make sure that you delete any indices which have been created with squirro_v8 template. 
# curl http://localhost:9200/_cat/indices -> then inspect for occurrences of squirro_v8 in the output.
# If your cluster has been created with Squirro 2.6.0, released on April 3rd 2018 or later, 
# you do not have to worry about this.
# To delete: curl -XDELETE http://localhost:9200/squirro_v8*
# To delete old jaeger indices: curl -XDELETE http://localhost:9200/jaeger*

systemctl stop elasticsearch

Upgrade every storage nodes (one by one) by running:

yum install java-1.8.0-openjdk
yum update java-1.8.0-openjdk
yum update squirro-storage-node-users

# Ensure that `path.data` variable is set in ES. If you have used Ansible in the past to deploy Squirro, you will be affected by this bug.
grep -q "^path.data" /etc/elasticsearch/elasticsearch.yml ||  echo "path.data: /usr/share/elasticsearch/data/" >> /etc/elasticsearch/elasticsearch.yml

# update elasticsearch
yum update elasticsearch

# Disregard the ES rpmnew warnings as they will be resolved by `squirro-storage-node` cluster

# update storage node
yum update squirro-storage-node

# wait for elasticsearch status to be green before continuing with next storagenode

Upgrade each cluster nodes by running:

yum install java-1.8.0-openjdk
yum update java-1.8.0-openjdk
yum update squirro-cluster-node-users
yum update squirro-*
squirro_start

Resolve all the `.rpmnew` files in `/etc/squirro`. This process involves merging the changes between the `.ini` (e.g., `storage.ini`) and '.ini.rpmnew`(e.g., `storage.ini.rpmnew`) files and then eventually deleting the `.ini.rpmnew` files. Finally, restart the services with merged `.ini` files (or just use `squirro_restart` to restart all services).