...

This release primarily focuses on a lot of bug fixes as a preparation to our Long term support release.

Fixes

...

Sped up Spacy document processing, required for the intelligent typeahead on item content

...

Visual fixes around item detail when body is empty.

...

Fixed the import of Cognitive search project template.

...

Fixed the Importing a project from a Squirro instance which does not have the refactored pipeline steps does not work

...

[Properies] Facets dropdown not scrollable

...

[Pipeline Editor] Step facet properties not being saved after changing value

...

Explore: Sometimes sub panel fails to render with entities selected

...

Remove Data URLs from emails other than Newsletter

...

"[Data Sources] ""Add feed source URL"" button has a weird circle animation"

...

"Pipeline editor - The link under Related in the left panel ""Add new Relate stepts in the AI Studio"" links a user to the 'dashboards' space"

...

[Pipeline] Pipeline edit middle draggable section frontend implementation

...

Pipeline testing

...

Unable to rename step in pipeline workflow

...

Extend translations support for Studio plugins

...

Extend translations support for Guides

...

Translate options for Frontend Connectors

...

Alignment/Overlap issues with some languages

...

"[Data Sources] ""Run"" on CSV and FEED data source returns 204 - no content"

...

[backend] Serialize the default_value of this kind of config option to a JSON formatted string

...

[backend] Enable modification of pipeline step names

...

"All existing KEE steps should be migrated to ""Relate"" automatically"

...

Content based typeahead - Caching key needs to add project identifier

...

Ground Truth list view: list of documents overflows

...

Item detail: x button is misaligned.

...

Project selector (name) is missing if there's only one dashboard

...

Ground Truth: labeling bigger documents triggers 414 error

...

Settings pages are missing space at the bottom

...

Setup experimentation project (index some 100k to million documents)

...

"[Explore Dashboard] Creating ""Favourite"" with alert returns 503"

...

Unable to create alerts

...

[Sprint goal] Finish full implementation for AI Studio feedback processing

...

Redesign frontend coding challenges

...

Design concept for selection of sentence splitting rule set

...

Implementation of sentence splitter picking concept

...

Use the new sentence splitting step in the DSS templates

...

Create rule set to split MRN data

...

Move sentence splitting logic out of integration sub module

...

Create advanced sentence splitting step in lib/nlp with addable rule option

...

"Gather other document styles, which cause issues"

...

Extend Ground Truth to store sentence splitting info

...

View for configuring a Ground Truth is not rendered properly

...

Data loader frontend config - Defaults don't show up

...

Data loader frontend config - booleans break frontend when mappings.json provided

...

Language switching in dashboards / frontend

...

Ability to rerun whole pipeline for already processed data

...

Pre-populate pipeline with default steps

...

Weird mixed state leads to inability to logout

...

Pipeline design review

...

Pipeline UX design

...

Salesforce SDK: Downloading fonts from Salesforce instead of Squirro

...

[Sprint goal] Migrate pipeline API for new frontend

...

[Sprint goal] Pipeline frontend phase 2

...

Salesforce SDK: Styles bleed from Squirro to Salesforce

...

Salesforce SDK: unable to open items

...

Cannot change avatar picture - save button missing

...

Search bar overlaps tabs on communities screen

...

Cognitive Search: PDF Highlighting of Query Matches not working

...

Fresh installation of filbeat prints out errors

...

[Sprint goal] Finalise Sales Insights UI

...

Update newsletter template to look like the original designs

...

Ensure that out of the box connectors are available on new installations

...

Upgrade Cognitive Search Template to support additional typeahead strategies

...

Brush up connector page

...

Adapt the SQ Dataloader behavior to not kill jobs automatically every 30 mins.

...

Newsletters: Item links should redirect the user to a squirro dashboard instead of original source

...

Type-ahead refresh for cognitive search

...

[Sprint goal] Better understand performance limitations of current query processing

...

Load activity_tracking.json logs from frontend service into elastic

...

Provide image uploading capability for GrapeJS email editing

...

Investigate performance of initial fuzzy auto-completer implementation

...

Triggering a sample email for testing purpose does not get populated with data.

...

Add support for PDF documents

...

Tests for newsletter sending

...

Community item count not displayed inside the card

...

Configure upgrades with Ansible Tower

...

Community list: sorted list should not be case sensitive

...

Have a dedicated endpoint to display a community

...

On getting the link to the detail page the overlay with the link is not shown

...

funny overlay in cognitive search

...

Search option on communities list page to find a community

...

Change default sort order for cognitive search to: “By Relevance”

...

Cognitive Search: Communities - Horizontal Scrolling and Search Bar overflowing community tabs

...

Recent Query Suggestor Service: Add Log extractor

...

Add infinity scroll loading for communities

...

Cognitive Search: Highlights from global feed -> only first item of the cards has community display -> for starred items

...

Write smoke-tests to preserve functionality after upgrades

...

Cognitive Search: Global Search Bar not taking right colours

...

[Sprint goal] Finish newsletter implementation

...

Migration scripts fail in cluster context

...

Item detail view: pixel perfection

...

"If you enable compact mode for the navbar and search using global search, the dashboard tabs move down"

...

Tab widget pixel perfection

...

AI Studio

Fixed an issue where

• Recommend bubble is not completely displayed and thumb-down button isn't aligned.

• There is a mis-configured link in the build-model step.

• Rejected feedback doesn't get marked as processed.

• Studio plugins in AI studio space fail.

• In the Feedback widget for incorrect classification, there is no option to select the correct class.

• View for configuring a Ground Truth is not rendered properly.

• Ground Truth: labeling of bigger documents triggers 414 error.

• List of documents overflows in the Ground Truth list view.

Cognitive Search

• Added a tooltip to see full name in communities.

• Search bar no longer overlaps tabs on communities screen.

• Fixed the import of Cognitive search project template.

• Improvements to PDF Highlighting of Query Matches.

• Upgrade Cognitive Search Template to support additional typeahead strategies.

• Change default sort order for cognitive search project template to: “By Relevance” from "By Time".

• Visual fixes around horizontal Scrolling and overflowing Search Bar community tabs.

• Highlights from global feed now has community display for all items instead of just the first item.

• Fixed an issue where Global Search Bar not taking right colours.

• Add a configurable option to disable number of items for communities if required. Turning this option will have a performance impact.

• Spacy NER cannot handle large text inputs which is used for content typeahead.

• Fixed an issue where Communities headline widget has a broken arrow shape.

• Fixed an issue where Favorite filters search is broken.

• Fixed an issue where community query displayed inside the breadcrumb instead of community name

• Cognitive search: Global search not showing typeahead suggestions

• Favorite filters search is broken.

• Studio plugins in AI studio space fail

• Pipeline Editor: Cannot delete workflow which was recently created and saved without reentering the editor

• Feedback AI Studio: Rejected feedback doesn't get marked as processed

• Communities headline widget: Arrow shape broken

• build Model: There is a miss configured link

• Incorrect textcolor

• Feedback AI Studio: Recommend bubble is not completely displayed and thumb-down button isn't alligned

• NLPTagger: Spacy NER cannot handle large text inputs

• Scheduled newsletters not sent on testing server

• Update scheduling frequency on project settings does not reflect in already scheduled newsletter tasks.

• [Projects Sidebar] Projects names are not visible

• "Pipeline editor: On using scroll arrows in the top left of the editor, the pipelines should scroll instead of jump"

• Refactor squirro-ansible upgrades into AWX

• "Feedback widget: incorrect classification, there is no option to select the correct class"

• Pipeline Editor: Displaying issue with the config options of the Near-Duplicate Detection

• Add an option to disable number of items for communities

• Pipelet implementing getArguments() is not rendered as 1st class widget -- throws error instead

• Some steps which are already part of a workflow are missing values for certain properties

• Provide a config key for every step returned by the GET /v0/projects/{project_id}/pipeline_sections endpoint

• Add support for multiple language files to the translator tool

• IB App UI / Dashboard development: The UI of the IB App requires a refinement to reflect business and usage requirements.

• Error when configuring source with pipeline steps that come from a DL plugin

• [Pipeline] Pipeline edit left sidebar frontend implementation

• Add mechanism that automatically removes the processed raw

• Added infinity scroll loading for communities.

• Added a Search option on communities list page to find a community.

• Sorted list of communities on communities list page is not case sensitive.

• Newsletter

  • Sending of newsletters does not fail when

    • There is no full name.

    • No communities are subscribed.

    • With additional_data error.

  • Update of scheduling frequency on project settings now reflects again in already scheduled newsletter tasks.

  • Triggering a sample email for testing purpose also gets populated with data.

  • Item links now redirect the user to a Squirro dashboard instead of the original source.

Translations

• We now also have support for French and Italian language in Squirro interface. Needs to be enabled in the configuration.

New Ingestion Pipeline Editor

The pipeline editor has been completely recreated. The new editor is more visual, and provides a much easier overview of the various pipelines in a project.

• In addition to that we have also laid down a lot of ground-work to allow for the re-running of pipeline workflows (for datasources running in the Frontend only) which allows for easier experimentation during the project setup process. For the more technical audience, this is enabled through the following underlying configurations. This will be included in the frontend in the upcoming releases.

  • We have added a new built-in pipeline step “Transform Input” which does the item fields and facets mapping. This was previously done in the dataloader itself but can now be handled in the pipeline itself. This step is controlled using the configuration option item_transformation_in_pipeline. It is disabled by default, and should be considered a beta feature for this release.

  • We have introduced a new processed directory in the Ingester to store the input data to this directory before performing pipeline steps. This enables us to keep a copy of the raw data to re-run the pipeline without fetching the data from the original source. This behavior is controlled by the configuration option keep_processed_data, which is also disabled by default.

  • We have also extend the Ingester to automatically remove the input data after a certain time period or disk space threshold

• Enable triggering the rerunning of a pipeline workflow on the already processed raw data

• Modify ingester/lib.stream to move the raw data to a processed directory after executing the pipeline when on dev mode

• Add configuration option that denotes whether we are running in development or production mode

• Add new built-in pipeline step Transform Input that does the work of mapping

• Remove validation logic regarding the mappings from the api.provider

• Remove the transformation logic of mapping source fields to item fields and facets from the Dataloader

• Add config_options to built-in steps

• Newsletter sending fails if there is no full name

• Newsletter test sending fails when no communities subscribed

• Newsletter test sending silently fails with additional_data error

• Cognitive Search: Communities - Add Tooltip to see full name

  • to avoid disk over-filling. This is controlled by the configuration options days_to_retain_processed_batchesand hours_to_retain_processed_batches. This mechanism kicks in when the keep_processed_data is enabled.

• In addition, we now offer three different Pipeline Workflow presets designed for various use-cases.

...

• We have also added the functionality to rename all of the steps in pipeline workflow to your liking.

• Pipelets which hint in their names that they perform Known Entity Extraction are now by default categorized in the “Relate” section of the Pipeline Editor.

Miscellaneous

• Projects names are visible again in the project sidebar.

• Adapt the SQ Dataloader behavior to not kill jobs automatically every 30 mins as long as we can fetch new data.

• Fixed an issue where user cannot change avatar picture because save button missing.

• Add animation for “Add feed sources” button on the feed dataloader plugin.

• Fixed an issue where Startup of topic service would fail to install saml2 plugin because of failing to install the pysaml2 dependency.

• Multiple Visual bug fixes around Cards widget and item detail view.

• Improve exception handing in the feed plugin.

• Scan endpoint now is not limited to 1000 entities per call/iteration.

• Fixed the creation of favorites on the dashboard.

Salesforce SDK

• Fonts are now downloaded from Salesforce instead of Squirro.

• Styles no longer bleed from Squirro to Salesforce.

• Fixed an issue where we were unable to open items.

Breaking Changes

• With the introduction of the new pipeline editor, the navigation structure in the Setup space has changed. To make room for the new pipeline editor to use the full width, all the options that were previously under the Enrich tab have now moved to the new AI Studio tab. Rerunning of enrichments will soon disappear from there and then move into the new pipeline editor properly. As a result, any custom studio plugins under these sections have to be re-uploaded to either one of the existing sections or to a new section called “AI Studio”. This Can be achieved by specifying the ”location”: “dss” in the studio_plugin.json file.

• Pipeline steps which are part of existing Pipeline workflows should continue function normally given that the included migration script was executed successfully. In case that something is not correct with an existing step of a workflow (e.g., no configuration options for a Pipelet when it should have), please remove the faulty step and add it again to the workflow. Then it should work as expected.

Fresh Installation Instructions

...