Introducing Squirro 2.5.0 - Birch

We're proud to release Squirro version 2.5.0 - codenamed Birch. Birch represents a big step forward for the Squirro user interface and the overall user experience, brings Smart Filters a step forward and offers significant improvements to search ranking, trend detection and more. Details below...

New Features

A Fresh Design

Clean User Interface

• Design that speaks: we introduced a new design language to Squirro based on Google's Material Design which results in a cleaner, clearer and more intuitive user interface. It also provides us a foundation for the future to ensure we have a consistent experience as Squirro evolves. We're sure you'll love it!
• Beautiful, simplified layout: Restyled UI components laid out with enough white space to offer you relaxed experience while using the product. App bar, lists, tables, forms and all other components have enough breathing space around themselves.
• Easier on the Eye: we've done a number of things to make it easier to read content in Squirro. We've introduced Roboto - a new font typeface - with many styles that allows us to better highlight and contrast words in the content, making it easier to read overall, especially on mobile devices. We've introduced subtler grey shading of the content that makes it easier to see which items are new and which you've already read. We've also improved the tag styling to make them clearer and more visible.
• Intuitive icons and actions: a new set of geometrical icons with a consistent look and feel that make Squirro more intuitive to use. In addition we've used outline icons reduce the feeling "visual heaviness" and placed a "floating action button" clearly and visibly throughout the app.
• A unique identity: just the right amount of Squirro-color applied across tabs, floating buttons, loaders and content highlighting to make a product that stands out.
• It's alive: by adopting Material Design CSS3 animations and motions we've been able to make the whole application come alive with subtle transformations, ripples, and ease outs, all of which help make user interface more intuitive.

Intuitive User Experience 

• Organised by role: to make it easier to find the features you need, we've organised features into what we call "Squirro Spaces". The first space - Dashboards - is where you can access graphs and visualisations you've created and easily share them with others. The second space - Setup - allows you direct access to Explore your project data as well as guiding you through the Squirro workflow, from loading data all the way through to visualising the results. The third space - Server - allows you to administer your Squirro instance managing which users and groups get access to which projects.
• Clear navigation: we've made it easier for you to find your way around Squirro. The app bar at the top helps you navigate between different projects, spaces and to user profile and settings. Below the app bar is the nav bar, which helps you move around within a chosen space. As you interact within a space "breadcrumbs" appear on the navigation bar to help you keep your place and the "floating action button" helps you focus when adding and editing entities. Overall we've laid out information in hierarchical order makes navigation and browsing easy.
• Fits better to your screen: by aligning content centrally in the Setup space, we're able to offer a better experience on all screen sizes. Tables and the forms have a maximum width that ensures they remain usable on really big screens. 
• Quick access to Trends and Smartfilters: we've organised trends and smartfilters so they're easy to find and edit inside of the Setup space.
• Easier to Explore: when Exploring data in your project, alongside Facets, Smartfilters and Favorites (previously called Saved Searches), we have a new section called Trends to make them easy to access in a search context. We've made applying these features to your search more intuitive and easier to control with checkboxes and cleaned up by restyling the facet list and fly-out. As you apply filters they are now "collected" on top of the filter panel, making it easy to adjust which filters are in play or remove them all at once.
• Friendlier Dashboard editing: when you enter dashboard editing mode, you'll find we've reduced the "visual noise" by hiding all features apart from the the editor tools. Together with the navigation bar this makes it easy to quickly jump in and out of editing mode. We've adjusted the colour scheme to help you focus on the dashboard itself and what will be shared with its end users. Widgets themselves are now packed into "Cards" making it easier to understand which information belongs to which widgets and bold titles on the widgets help draw the users attention to more important information.
• Wonderful Widgets: we've also brought many elements of the new design to our library of widgets, such as ensuring the vertical and horizontal widgets employ the same style, applying consistent design across the "Reset" widget and the filters in the Explore screen and applying an attractive new style to the time selection.

Smart Filters

Enhancements

• Part of speech tagging: for each extracted Smart Filter feature the part of speech is detected and you can now exclude not only individual features but a whole group of features of the same kind. E.g. exclude all adverbs from being part of your Smart Filter.
• Feature learning: we're excited to be able to add (experimental) support for the word2vec and fastText algorithms. This enables Smart Filters not only to identify existing significant terms in training documents but also suggest related terms without you needing to provide them.
• Easier manual configuration: the manual Smart Filter editor now understands stemming so you can enter "house" or "houses" and still get the same results. We've also added wildcard support to you can specify terms like Polit* and match words like "Political", "Politics" and "Politico"
• Lots of bug fixes and tuning: including...
  • Matched and highlighted tokens are now explained when using manual smart filter with near-query operators or when using single non-quoted UI token representation names
  • "Convert to Manual" no longer produces invalid Smart Filters
  • Apostrophe are no longer considered as as part of a token, so terms like "this year" and "this year's" would both be matched by a rule contain Year.
  • Space characters in the name no longer cause new Smart Filters to break when saving
  • Read-only users can create temporary smart filters again
  • Highlighting of smart filter matches in PDF documents now works correctly.

Improvements

• Scalable Trend Detection: trend data is now stored in it's own elastic search index, replacing the existing CSV file solution, and providing more powerful ways to access and aggregate trend data. Anomalies themselves have become a "first class citizens" in this index and we've optimised the approach we use to detect incoming anomalies allowing trends to perform more efficiently.

• Factoring Time into Relevance Ranking: when search results are ordered by "Relevance" we now include time as a component, making it possible to prioritise "newness" over straight matching of items. This is particularly powerful when incoming data contains important news - the latest items are frequently much more important than a good match that's months old. You can fine tune this relevance factor using the standard Squirro configuration files - see Configuration for "Elasticsearch" settings.

• Fast Multiple Trends: it's now easier to present multiple trends in a single dashboard without performance issues. More powerful visualisations of multiple trends coming soon...
• Python Updated: Squirro now supports (and requires) Python version 2.7.13
• KEE Specification: added support "invalid" arrow syntax in JSON file
• Smart Elasticsearch Logging: elasticsearch.trace can now be included in log file output (if configured). It's now also possible to override the overriding elasticsearch client log level
• Cluster Monitoring: 503 exceptions are now handled by the heartbeat monitor and log handlers have been added to Kazoo logging, providing greater transparency over cluster state
• Native Elasticsearch Client: we replaced our internal Elasticsearch client with the native Elasticsearch client enabling performance increases and access to specific features related to bulk handling of items
• Flawless source deletion: when deleting a source/subscription, Squirro now avoids showing any intermediate state.

Bug Fixes

• Fixed issue with pivot data handling in the dataloader

• JSON encoding issues from data provides are now better handled
• Disable trend chart data grouping when using time selector
• Allow redis to bind on unreserved ports as well with selinux
• Modifying a subscription no longer leaves an old source behind in the DB