Dashboard/Widget Query Parametrisation:

• Dashboard/Widget Query Parametrisation enables the use of dashboard store parameters to pass additional queries to selected widgets.
• Basic functionality:
  • Add <%= store.key %> to the "Additional Widget Query"
  • The value stored in store.key is then added as an additional query to this widget.

  • The Dashboard Store has been exposed, you can store defaults to be used with the Widget Query Parametrisation.

Enablement for Global Search / Master Dashboard / Conversational UI  

• Global Search
  • Global Search is placed in the app bar only for the dashboard space. It runs a search on the Master dashboard (see below) to quickly find app content.
  • Global search query can be a normal search or a NLP query to search any result across the app which technically will be a search inside the Masters Dashboard.
  • It will not filter down the active dashboard tab as it is supposed to search for results not readily placed under the visible dashboards.
  • A user can type a query, upon pressing enter, the search is submitted. Any layer/s from the master dashboard that matches the visibility condition is made visible.
• Master Dashboard
  • Master dashboard is essentially a templating dashboard where by using layers, one creates visual templates for different query parameters (including NLP) and visibility conditions.
  • This dashboard is not designed to be any of the main dashboards shared with the end-user, instead it is supposed to show layers which meet a visibility condition caused by an NLP or normal query from the Global Search bar.
  • By default, a project does not have a dashboard marked as master but any dashboard in a project can be marked as master.
  • Only one dashboard at a time can be a master.
  • Master dashboard is not visible in the Dashboard space.
  • This dashboard controls the visibility of the Global Search bar in the app bar. If a dashboard is marked as master, Global Search bar will appear, otherwise not.
• NLP
  • We have built the foundation for a new feature we call Conversational UI. This enables an NLP driven search.
    
  • NLP queries are enabled from the Global Search bar and display the results in the consumption mode.
  • A Master Dashboard is built with layers as templates designed to be visible for all combinations of NLP parameters. These layers are displayed as NLP query results.
  • In order to answer to the NLP queries, the master dashboard layers should have NLP aggregations rules defined in the visibility condition.
  • It is also possible for a layer to be configured to 'Use NLP if available'. In that case, any manual configuration of the widgets inside that layer is ignored and the widgets are automatically configured to answer to NLP queries.
  • This feature is still highly experimental and therefore disabled by default. To enable it a few setup and configuration steps are required. Please reach out to Squirro if you are interested in an early access to this feature.
• Voice Search
  • Voice search is an experimental feature enabled from the Global Search bar and only supported on Chrome browser for now.

Data Sources

• JSON data loader
  
  • We have introduced a new JSON data loader via the frontend. It is placed in the Data Import section.
    Image Removed
  • The dataloader supports:
    • Files with single JSON object (auto-detect)
    • Files with a list of JSON objects (auto-detect)
    • Files with nested JSON objects, i.e. the JSON objects are not top-level. For this, specify the item-schema option in dataloader
    • Directories (only via CLI)
  • See documentation on how to use it via CLI
• Addition of static keyword for data source on Frontend
  • Option to add keywords to all items coming from a particular data source during the data loading process.
    On Step 4: 
    Image Removed
• Behaviour of duplicated source changed
  • When duplicating a source, the source is now in "paused" state, which allows editing the data source configuration before starting the data load by "Resume".

Filtering and Pagination on User Management Page

• User management on the server now enables searching for users and a pagination functionality.
• Search functionality was also added to Project Member page.

Image Removed

Project Export/Import of Squirro Items

...

APP & Nav Bar Theming

• Please jot down points from here: https://nektoon.atlassian.net/wiki/spaces/DES/pages/878510205/App+Nav+Bar+Theming
  Image Removed

Widgets & Dashboard

New Widgets

• 3D Scene
  
  • Example with 3 different rotations of the 3D object. Object can be rotated, zoomed in/out, moved around.
  • Colour overlay is also possible.
    Image Removed
  • For STL files, rotate the object and visualise it in 3D. 3D is displayed at the top of the item detail. Addition of 3D icon in item toolbar with ‘Scroll to’ functionality.

Widget Improvements and Changes

• Card Widget:

  • "Grid" Mode added: → responsive mode that has vertical scrolling and shows different number of cards from left to right depending on width of available space.
• Table widget:
  • "Option to select multiple values in table widget. The values selected are "OR"-ed instead of "AND"-ed.
• Line Chart:
  • Line Chart widget enables now the specific labelling of the Axes.
    Image Removed

• Result List Widget
  • Copy to clipboard option added to the item detail source info. in the metadata.
     Image Removed
  • In the list view, if the URL exists for an item, the URL domain is displayed instead of the source name.

• Link Widget
  • The link widget was extended so that instead of a link text, one can show either Materialize Icons or any other Icon/Image by providing a URL.
    Image Removed

Improvements

Dashboard Permissions:

• Provision of fix that dashboard permissions are properly handled during Project Export/Import:
  • Unrestricted dashboards remain unrestricted
  • Permissions on restricted dashboards are recreated to only allow admins on the new project to view it (since users on the original project might not be part of the new project)
• Membership changes on tenant level (Admin, User, Restricted) are propagated into dashboard permissions

Pipelet running as part of dataloading

• Allow running the same pipelet multiple times with different configuration
• Instead of using class name as pipelet key, use new field: ‘class_name'. See documentation. 

Data loading UI

• Duplicating a data source will not automatically kick off the job, but rather put it in 'paused' state, allowing for possible modifications before loading.

Pipelet rerun

• About 2x speed improvement
• Remove the 10,000 items limit → able to use on any number of squirro items

LineChart API improvement

• Extended the time_selection range to support per-second aggregation (time_selectiion=Xsecond).
• Better management of the query_aggreation when time_selection is not present in the query.

Other Improvements

• Update of several libraries to the latest versions
• Feed (RSS or Atom) plugin now allows for custom date field & format
• European Patent Office dataloader revived & updated

Bug Fixes & Improvements

...

...

Dashboard/Widget Query Parametrisation:

• Dashboard/Widget Query Parametrisation enables the use of dashboard store parameters to pass additional queries to selected widgets.
• Basic functionality:
  • Add <%= store.key %> to the "Additional Widget Query"
  • The value stored in store.key is then added as an additional query to this widget.

  • The Dashboard Store has been exposed, you can store defaults to be used with the Widget Query Parametrisation.

Enablement for Global Search / Master Dashboard / Conversational UI  

• Global Search
  • Global Search is placed in the app bar only for the dashboard space. It runs a search on the Master dashboard (see below) to quickly find app content.
  • Global search query can be a normal search or a NLP query to search any result across the app which technically will be a search inside the Masters Dashboard.
  • It will not filter down the active dashboard tab as it is supposed to search for results not readily placed under the visible dashboards.
  • A user can type a query, upon pressing enter, the search is submitted. Any layer/s from the master dashboard that matches the visibility condition is made visible.
• Master Dashboard
  • Master dashboard is essentially a templating dashboard where by using layers, one creates visual templates for different query parameters (including NLP) and visibility conditions.
  • This dashboard is not designed to be any of the main dashboards shared with the end-user, instead it is supposed to show layers which meet a visibility condition caused by an NLP or normal query from the Global Search bar.
  • By default, a project does not have a dashboard marked as master but any dashboard in a project can be marked as master.
  • Only one dashboard at a time can be a master.
  • Master dashboard is not visible in the Dashboard space.
  • This dashboard controls the visibility of the Global Search bar in the app bar. If a dashboard is marked as master, Global Search bar will appear, otherwise not.
• NLP
  • We have built the foundation for a new feature we call Conversational UI. This enables an NLP driven search.
    
  • NLP queries are enabled from the Global Search bar and display the results in the consumption mode.
  • A Master Dashboard is built with layers as templates designed to be visible for all combinations of NLP parameters. These layers are displayed as NLP query results.
  • In order to answer to the NLP queries, the master dashboard layers should have NLP aggregations rules defined in the visibility condition.
  • It is also possible for a layer to be configured to 'Use NLP if available'. In that case, any manual configuration of the widgets inside that layer is ignored and the widgets are automatically configured to answer to NLP queries.
  • This feature is still highly experimental and therefore disabled by default. To enable it a few setup and configuration steps are required. Please reach out to Squirro if you are interested in an early access to this feature.
• Voice Search
  • Voice search is an experimental feature enabled from the Global Search bar and only supported on Chrome browser for now.

Data Sources

• JSON data loader
  
  • We have introduced a new JSON data loader via the frontend. It is placed in the Data Import section.
    Image Added
  • The dataloader supports:
    • Files with single JSON object (auto-detect)
    • Files with a list of JSON objects (auto-detect)
    • Files with nested JSON objects, i.e. the JSON objects are not top-level. For this, specify the item-schema option in dataloader
    • Directories (only via CLI)
  • See documentation on how to use it via CLI
• Addition of static keyword for data source on Frontend
  • Option to add keywords to all items coming from a particular data source during the data loading process.
    On Step 4: 
    Image Added

Filtering and Pagination on User Management Page

• User management on the server now enables searching for users.
• A pagination functionality for server users was also implemented.
• Search functionality was also added to Project Member page.

Image Added

Project Export/Import of Squirro Items

• Project Export/Import enables now the export of items from a Squirro Project and the import in a new Squirro Project.
  (Export/Import of binary documents is not supported.)
• The data imported like this will all be added as a single source called "Project Import".
• This functionality facilitates the migration of projects between servers.
  
  Image Added

APP & Nav Bar Theming

• With the new theming functionality, essential branding and theming can be done on the frontend.
• Especially the app and the navigation bar sitting on the top of the dashboards shared with the end-user can be styled here.

• Functionalities:
  • Custom Logo to replace Squirro logo next to Project Name
    • Set custom logo via image-URL
    • max. size of logo should be 50 kb to render smoothly.
  • Set App / Nav bar background colour
  • Set App / Nav bar text colour
  • Set Brand colour
  • Set Highlight colour
  • Colours can be defined in the following formats:
    • Hex Code
    • Short Hex Code
    • RGB %
    • RGB Absolute
    • keyword (eg. orange)
      
      Image Added

Widgets & Dashboard

New Widgets

• 3D Scene
  
  • Example with 3 different rotations of the 3D object. Object can be rotated, zoomed in/out, moved around.
  • Colour overlay is also possible.
    Image Added
  • For STL files, rotate the object and visualise it in 3D. 3D is displayed at the top of the item detail. Addition of 3D icon in item toolbar with ‘Scroll to’ functionality.

Widget Improvements and Changes

• Card Widget:

  • "Grid" Mode added: → responsive mode that has vertical scrolling and shows different number of cards from left to right depending on width of available space.
• Table widget:
  • Option to select multiple values in table widget. The values selected are "OR"-ed instead of "AND"-ed.
• Line Chart:
  • Line Chart widget enables now the specific labelling of the Axes.
    Image Added

• Result List Widget
  • Copy to clipboard option added to the item detail source info. in the metadata.
     Image Added
  • In the list view, if the URL exists for an item, the URL domain is displayed instead of the source name.

• Link Widget
  • The link widget was extended to support having links with icons instead of text based links. The supported icons are the Material Design Icons.
    It is now also possible to add multiple links to the same widget. Learn More
    Image Added

Improvements

Squirro CLI Developer commands:

• On a Squirro server running 2.6.5 you can now execute the following commands:

Dashboard Permissions:

• Provision of fix that dashboard permissions are properly handled during Project Export/Import:
  • Unrestricted dashboards remain unrestricted
  • Permissions on restricted dashboards are recreated to only allow admins on the new project to view it (since users on the original project might not be part of the new project)
• Membership changes on tenant level (Admin, User, Restricted) are propagated into dashboard permissions

Pipelet running as part of dataloading

• Allow running the same pipelet multiple times with different configuration
• Instead of using class name as pipelet key, use new field: ‘class_name'. See documentation. 

Data Source Duplication

• When duplicating a source, the source is now in "paused" state, which allows editing the data source configuration before starting the data load by "Resume".

Pipelet rerun

• About 2x speed improvement
• Remove the 10,000 items limit → able to use now any number of squirro items.

LineChart API improvement

• Extended the time_selection range to support per-second aggregation (time_selectiion=Xsecond).
• Better management of the query_aggregation when time_selection is not present in the query.

General Widget API Improvements

• Recommendations widget can now support deselecting the recommendation
• Exposed shouldUseBaseCollection in the Recommendations Events widget, allowing it to switch to standard Items collection dynamically
• Dashboard Loaders now also run in edit mode
• Disable Drilldown is now supported in the Recommendations widget
• Extended getCustomItemDetailOptions in Result List (Search) widget to support custom icons

Other Improvements

• Update of several libraries to the latest versions
• Feed (RSS or Atom) plugin now allows for custom date field & format
• European Patent Office dataloader revived & updated

Bug Fixes & Improvements

Added on October 6, 2019 (build 139 - patch 2)

• Implement multi-value selection for Facets List Widget
• Fix for custom widget issue found on safefood.ai
• Fix _selectedTargetValue needs to be a string to work
• Replacing long URLs with only the domain
• GET endpoint for login resource (to be used by Squirro SDK)
• Extend recommendation API
• Fix Facets list widget disable drill-down mode
• Add PyJWT dependency package by default

Added on October 6, 2019 (build 146 - patch 7)

• Additional fix for long source names / PDF issues
• Disable Newrelic by default
• Fix issue at SCB scrolling on entity view
• Only show search if more than 15 projects
• Disable NewRelic
• Add helper methods to recommendations cocktail

Added on October 10, 2019 (build 144 - patch 15)

• Do not limit Link Widget to 4 columns of width
• Add toggle to Link Widget
• Fix grid throw after Link Widget change
• Fix ML Jobs export

Added on November 6, 2019 (build 158 - patch 17)

• Project reader user is now allowed to star items
• Expose multi link property to Widgets API
• Fix for App/Nav bar theming issue on dashboard space
• Fix for topic cluster (honey comb) widget

Added on November 6, 2019 (build 159 - patch 18)

• Fix autosize to fit frame
• Remove PPIQ connector
• Entity tooltips rework
• Fix section reset
• SDK: Fix login
• Add logout to sdk destroy

Added on November 18, 2019 (build 166 - patch 20)

• Scan returns entities now also on the second and following request
• Disabled legacy KEE highlighting
• Fix Facets List on IE
• Fix & issue in the entity view

Added on November 21, 2019 (build 168 - patch 22)

• Fix typeahead visibility

Added on November 21, 2019 (build 169 - patch 23)

• Allow userPref widget to support arrays over strings
• Item detail toolbar API extensions
• Handle lists in merge step
• Install feed parser for feed plugin by default
• Templating for IFrame and RTE widgets
• Fix item detail paragraphs
• Fix feedback format

Added on December 3, 2019 (build 170 - patch 24)

• Search Bar Widget rework
• Avoid rendering item detail on window resize
• Fix result list issue when very long words/links are present

Added on December 5, 2019 (build 171 - patch 25)

• Multiple improvements to lib/nlp
  • <pre> tags in HTML delimit paragraphs
  • Performance fix for doc_split_filter
  • Handle newline in HTML tokenizer
  • Fix any-order matches in proximity filter. The words can now be present in any order at all.
  • Fix proximity filter for one-words. The words must be entire words, not simply substring matches.

Added on December 11, 2019 (build 174 - patch 26)

...

• Update JRE to latest version (8u231)
• Fix bug where clicking outside dropdowns does not closes it
• Fix multiple bugs on Facet List widget
• Link widget delete icon issues
• Additional fixes to the User Preferences widget

Added on December 17, 2019 (build 175 - patch 27)

• Avoid render when bookmarking an item

Added on February 5, 2020 (build 176 - patch 28)

• SB-398: Fixing recommendation input dropdown
  
• SB-384: Reset Widget - applied filters modal does not disappear when you remove all the filters
• SB-377: Filtering step can not handle Unicode title
• Improving entity highlighting in item view
• Adding support for id.squirro.com

Added on February 12, 2020 (build 177 - patch 29)

• Advanced entity feedback for recommendations

Added on February 18, 2020 (build 178 - patch 30)

• SQ-10880 - Result List Widget: When only "Show entities" selected, sometimes need to reselect entities
• SB-313 - Can't type the letter s in the html widget because s seems to be a shortcut for starring an item
• SB-395 - Rich Text Editor Widget content is not scrollable anymore
• SB-415 - Incorrect classification Feedback popup is not mapping the catalysts correctly
• SQ-11559 - Unify payload sent for up/downvoting and feedback
• SB-416 - Selection tagging is removing consecutive spaces from sentences
• SB-413 - User page option to change "Rows per page" number doesn't work
• SB-407 - Rerunning on items with entities fails

Added on April 25, 2020 (build 178 - patch 31)

• Switch from Oracle JRE to latest iteration of openjdk-1.8 (242).
  • Please refer to the updated ugrade instructions below.
• Update python2 to version 2.7.17.
• Update python3 to version 3.6.9.

Installation & Upgrade

Fresh Installation Instructions

...

yum install java-1.8.0-openjdk (If not already installed)
yum update java-1.8.0-openjdk
yum update squirro-storage-node-users
yum update squirro-storage-node
yum update squirro-cluster-node-users
yum update squirro-*
yum erase jre

yum install java-1.8.0-openjdk (If not already installed)
yum update java-1.8.0-openjdk
yum update squirro-storage-node-users
yum update squirro-storage-node
yum erase jre

yum install java-1.8.0-openjdk (If not already installed)
yum update java-1.8.0-openjdk
yum update squirro-cluster-node-users
yum update squirro-*
yum erase jre

yum install java-1.8.0-openjdk (If not already installed)
yum update jrejava-1.8.0-openjdk
yum update squirro-storage-node-users
yum update squirro-storage-node
yum erase jre

yum install update jrejava-1.8.0-openjdk (If not already installed)
yum update java-1.8.0-openjdk
yum update squirro-cluster-node-users
yum update squirro-*
yum erase jre