...
[Query Processor] Create a background task that will keep default query workflow up to date. It makes that all changes and improvements to the query processing will be always reflected in the already created workflows.
[Query Processor] Don't import default query processing workflows because they are created automatically during the project creation. It prevents having multiple default workflows in the same project.
[Query Processor] Create query processing workflows in all projects. It assures that all projects that are missing default query processing workflow, will have it from now on.
[Search]
preserve_scroll_order: Parameterize scan API to choose if index-scrolling should apply provided sort-order (expensive operation) or no sort at all(return documents from index order, fast & lightweight operation).[Search] Add preserve_order parameter to
squirro_client.scan()method (perform deep pagination with items considering sorting order).[Search] Refactored/simplified handling of synonym search to support synonyms on managed ES (no usage of synchronized synonym files across the cluster, but usage of inline ES settings).
[Typeahead] Merge similar suggestions returned from typeahead.
[Typeahead] Remove punctuation from content based typeahead.
[Database] Extend possible package size sending to MariaDB server. It resolves problems when big amount of data (like file with synonyms) breaks the SQL server. For more information, please refer to this page.
[Pipeline Steps] The
Change Pipelineoffers the option to configure the batch priority of the generated batches. The default priority is low for the batches generated from this step.[Pipeline Steps] A
PDF Conversionstep was added. Now incoming Office documents can be converted into PDF files, thus enabling better rendering in frontend and some more sophisticated processing in the backend.[Platform] A new
PDF Conversionservice was added. It behaves and can be controlled similarly to the existing Squirro services that comprise the Squirro platform. The purpose of this service is to handle the conversion of documents to PDFs.[Platform] The
PDF Conversionservice removes unused PDF renditions after some time to prevent exceeding storage capacity.[Platform] FileContainer now uses hardlinks for duplicated files to prevent excessive disk space usage.
[Data Ingestion] Add retry logic in the dequeue operation of inputstream batches. The maximum number of those retries is controlled by the server config option ingester.stream.max-dequeue-retries (default value is 3).
[Dataloaders] Support deletions in OneDrive & SharePoint dataloaders.
...