Page Comparison

...

Pipelets are written in Python. They need to inherit from the squirro.sdk.PipeletV1 class and implement the consume method. The simplest possible Pipelet pipelet looks like this:

from squirro.sdk import PipeletV1
 
class NoopPipelet(PipeletV1):
    def consume(self, item):
        return item

...

Configuration

When adding enrichments in the user interface, the administrator can pass in configuration to the pipelet. For example the title modification pipelet could accept a custom suffix that is to be added to the title.

The configuration for this is provided in as JSON data structure:

Image Added

This full data structure is then passed in to the pipelet constructor, where it can be retrieved. Usually it's then simply stored in a object variable so it can be used in the consume method again.

from squirro.sdk import PipeletV1

DEFAULT_SUFFIX = ' - Hello, World!'

class ModifyTitlePipelet(PipeletV1):
    def __init__(self, config):
        self.config = config

    def consume(self, item):
        suffix = self.config.get('suffix', DEFAULT_SUFFIX)
        item['title'] = item.get('title', '') + suffix
        return item

In this example, when the suffix hasn't been provided, the default suffix is used.

Documentation

A pipelet class can be documented using docstrings. The first sentence (separated by period) is used as a summary in the user interface. All the remaining text is used as a description and is often used to document the expected configuration. The description is parsed as Markdown (using the CommonMark dialect). The 60-second overview serves as a good reference.

from squirro.sdk import PipeletV1

DEFAULT_SUFFIX = ' - Hello, World!'

class ModifyTitlePipelet(PipeletV1):
    """Modify item titles.

    This appends a suffix to the title of each item. When no suffix is
    provided, it appends the default suffix of "- Hello, World!".

    Configuration keys:

    - suffix: The suffix to add to the title

    Example configuration:

        {"suffix": " - the pipelet ran"}
    """
    def __init__(self, config):
        self.config = config

    def consume(self, item):
        suffix = self.config.get('suffix', DEFAULT_SUFFIX)
        item['title'] = item.get('title', '') + suffix
        return item

Development Workflow

For developing pipelets, Squirro provides the pipelet command line tool as part of the Toolbox.

...