Page Comparison

...

Its syntax strongly resembled Python, but the complexity and functionality has been reduced. On the other hand Squirro Script is optimised for dealing with Squirro items, especially around how functions are used on keywords and the functions that are provided. For cases where Squirro Script is not powerful enough, pipelets can be used to use the full power of Python instead.

Table of Contents

Cookbook

For examples of Squirro Script, please refer to the Squirro Script Cookbook.

Using Squirro Script

Squirro Script is available in the server-side pipeline as an enrichment type. As Squirro Script is itself implemented as a pipelet, it can also be used in the data loader using the pipelets.json file.

Server-side Enrichment

To enable Squirro Script, the supporting pipelet needs to be installed first. In the latest beta, please use the Plugin Repository for this.

Once set up, add a Squirro Script enrichment to the project and use the following syntax to add a script:

{
    "commands": "hours = minutes / 60"
}

Note: this will be simplified in a future Squirro release to not have to enter the JSON boilerplate.

Data loader pipelet

Use the pipelets.json file of the data loader and remember that this file has to be indicated to the data loader with the --pipelets-file argument. An example pipelets.json file with Squirro Script looks like this:

{
    "SquirroScriptPipelet": {
        "file_location": "squirro_script.py",
        "stage": "before templating",
        "config": {
            "commands": '''
                duration_hours = duration / 60
                if duration_hours > 1:
                    long_meeting = 'true'
                else:
                    long_meeting = 'false'
            ''',
        }
    },
}

As you can see, this takes advantage of the Hjson multi-line string support. Open and close the string with three single quotes on each side.

Language Primer

Squirro Script is modelled very closely after Python with only small syntax additions. Note however, that compared to Python only very limited functionality is available.

Example

An example script could look like this:

...

This simple script calculates the hours keyword from the minutes keyword. It then shortens the title to a most 40 characters with an ellipsis.

Variables

There are three types of variables available. The Item Format documentation contains full details on the possible fields and keywords available.

...

Variable can be assigned with the = operator

Data types

There are four types of variables:

...

The types correspond to the type of Squirro facets The type of a variable is implicitly defined by the assigned value, or explicitly by using one of the to_* functions.

Literals (values)

Squirro Script supports strings and numbers. Strings are always assumed to be Unicode and are assumed to be UTF-8 encoded. Numbers can be full numbers (int) or contain a decimal separator (float). Examples: 42 or 3.1415926).

Conditions

Conditional logic can be implemented with the if block. Example:

...

For comparison the operators== (equal), != (not equal), '>' (greater than), '>=' (greater or equal), '<' (less than) and '<=' (less or equal) are available. If no comparison operator is used (such as if $title:), then the variable is treated as true if it is non-empty and not zero.

Operators

Operators can be used anywhere a variable can be referenced. The two sides of the operator can be any function call or variable. For example:

...

• +: for addition or concatenation.
• -: for subtraction of numeric values.

• *: for multiplication of two numeric values.
• /: for division of numeric values.

Lookup Tables

Lookup tables are used to link data in processed Squirro items to some other database. For this purpose lookup tables are provided in Squirro Script. When you need more matching capabilities, Known Entity Extraction is another way to go.

...

{
    "title": "Example",
    "keywords": {
        "products": ["firefox"],
        "category": ["browser", "software"],
        "vendor": ["Mozilla"]
    }
}

Functions

Functions can be called anywhere a variable can be referenced. For example:

...

attendee_count = count(attendees)
attendee_len = len(attendees)

{
  "title": "Example",
  "keywords": {
    "attendees": ["Jon", "Abigail", "Mark", "Rose ", "Isabella"]
  }
}

{
  "title": "Example",
  "keywords": {
    "attendees": ["Jon", "Abigail", "Mark", "Rose ", "Isabella"],
    "attendees_count": [5],
    "attendees_len": [3, 7, 4, 4, 8]
  }
}

String Functions

find(regexp[, field…])

Returns matches of the given regular expression in the input item and return any matching groups.

...

Removes all nodes that match the given XPath expression from the item body.

Numeric Functions

abs(num)

Returns the absolute value of the number.

...

Converts the given value to an int.

Date Functions

datediff(date1, date2[, unit="seconds"])

...

Converts the given value to a datetime. This only works if the string uses the Squirro datetime format (YYYY-MM-DDTHH:mm:ss).

Aggregation Functions

Aggregation functions are used on keywords and work on the entire list, instead of individual values. Most of the functions assume numeric values in the list.

...

minutes_len = len(minutes)
minutes_count = count(minutes)
minutes_max = max(minutes)
minutes_min = min(minutes)
minutes_sum = sum(minutes)
minutes_avg = avg(minutes)

Lookup Table Functions

extend_keywords(key_value_dict)

...

Looks up the given key or keys in the lookup table. Returns a list of all the matching entries.

Other Functions

clear(variable)

Clears the given variable. This is usually used on facets or item fields to clear them.

...