squirro_asset Command Line Reference

The squirro_asset tool can be used to manage all Squirro code artefacts called custom assets. Assets are Pipelets, Custom Widgets, and Custom Dashboard Loaders.

Table of Contents

squirro_assets tool

You can use the squirro_asset tools to manage all the custom "code artifacts" in Squirro server systems such as Dashboard Loaders, Dashboard Widgets, or Pipelets.

The first parameter is the type of asset: dashboard_loader, widget, or pipelet, and if at any time you are unsure of what parameters are allowed, you can request help by adding the --help parameter.

> squirro_asset --help
usage: squirro_asset [-h] [--verbose] [--log-file LOG_FILE] [--version]
                     {widget,pipelet,dashboard_loader,dataloader_plugin} ...

Squirro asset management tool

positional arguments:
  {widget,pipelet,dashboard_loader}
                        asset type help
    widget              Manage custom widgets
    pipelet             Manage custom pipelets
    dashboard_loader    Manage custom dashboard_loader
    dataloader_plugin   Manage custom dataloader_plugin

General Options:
  -h, --help            Show this help message and exit.
  --verbose, -v         Show additional information.
  --log-file LOG_FILE   Log file on disk.
  --version             show program's version number and exit

Dataloader plugin

You can upload new or modified dataloader plugins using the upload command, for example:

> squirro_asset dataloader_plugin upload --folder <dataloader-plugin-folder> --token <token> --cluster <cluster-url>

where <dataloader-plugin-folder> is a directory containing the following files: 

  • Dataloader plugin python file
  • A `requirements.txt` listing any external python dependencies needed for the plugin
  • A png file to be used as a logo for the dataloader plugin in the Squirro UI
  • A `dataloader_plugin.json` file containing the proper metadata for the dataloader plugin (see format below) in hjson format

Format of dataloader_plugin.json file. This file is parsed as HJSON, and can thus contain comments and trailing commas.

{
    # title of the dataloader plugin to be displayed in the UI
    "title": "Hipchat",

    # description of the dataloader plugin to be displayed in the UI
    "description": "For importing data into hipchat",

    # Name of the plugin file
    "plugin_file": "hipchat.py",

    # Name of the requirements file containing the python dependencies
    "requirements_file": "requirements.txt",

    # Category under which to show this dataloader plugin, Possible options are:
    # 'web', 'socialMedia', 'import', 'developers', 'enterprise', 'businessIntelligence', 'crm', 'itsm'
    "category": "developers",

    # Name of the PNG file to be used as a thumbnail while displaying the dataloader plugin in the UI
    "thumbnail_file": "Hipchat_Atlassian_logo.png"
}

You can list all dataloader plugins using the list option

> squirro_asset dataloader_plugin list --token <token> --cluster <cluster-url>
[
    {
        "requirements_file": "requirements.txt",
        "category": "developers",
        "config_options": [
            {
                "required": true,
                "name": "hipchat_url",
                "help": "URL of HipChat public web interface."
            },
            {
                "name": "hipchat_endpoint",
                "help": "Endpoint of the HipChat API (URL)"
            },
            {
                "required": true,
                "name": "hipchat_authtoken",
                "help": "Authentication token for HipChat API"
            },
            {
                "default": 3,
                "type": "int",
                "name": "hipchat_conversation_length",
                "help": "Number of days back into the past for which to fetch the message archive counting back from yesterday"
            },
            {
                "default": 1,
                "type": "int",
                "name": "hipchat_conversation_age",
                "help": "Number of days back into the past for which to fetch the message archive counting back from yesterday"
            },
            {
                "name": "hipchat_room_blacklist",
                "help": "Space-separated list of rooms to ignore"
            },
            {
                "default": "message",
                "name": "hipchat_mode",
                "help": "Whether to individual messages (realtime) or groups conversations (older than 1 day)"
            }
        ],
        "hash": "d8c6ffe1-1886-11e7-a5a4-f45c89acc4cb",
        "description": "For importing data into hipchat",
        "title": "Hipchat",
        "pipelets_files": [
            "keywords_pipelet1.py",
            "sentiment_pipelet2.py"
        ],
        "thumbnail_file": "Hipchat_Atlassian_logo.png",
        "name": "hipchat_thumbnail",
        "body_template_file": "body.j2",
        "thumbnail_b64_encoded": "+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+",
        "plugin_file": "hipchat.py",
        "resources": {
            "txt": [
                "requirements.txt"
            ],
            "py": [
                "keywords_pipelet.py",
                "hipchat.py",
                "load.py"
            ],
            "pyc": [
                "hipchat.pyc"
            ],
            "png": [
                "Hipchat_Atlassian_logo.png"
            ]
        },
        "title_template_file": "title.j2"
    }
]

Dashboard loaders

You can upload new or modified dashboard loaders using the upload command, for example:

> squirro_asset dashboard_loader upload --folder <dashboard-loader-folder> --token <token> --cluster <cluster-url>

where <dashboard-folder> is a directory containing dashboard loader html, css and javascript files along with a file named dashboard_loader.json formatted in hjson, and you specify one of the options to connect to a Squirro server installation (using <token> and <cluster-url> in the above example).

You can list all dashboard loaders using the same parameters as for upload except --folder:

> squirro_asset dashboard_loader upload --token <token> --cluster <cluster-url>
[
    {
        "hash": "ec43609e-c5c6-11e6-ac8a-80e65013fba6",
        "name": "name-1",
        "title": "title-1",
        "directory": "/var/lib/squirro/topic/assets/dashboard_loader/tenant",
        "resources": {
            "css": [
                "mycss.css"
            ]
        }
    },
    {
        "directory": "/var/lib/squirro/topic/assets/dashboard_loader/tenant",
        "hash": "bed8e082-c395-11e6-94ff-000c29dfbc49",
        "name": "name-2",
        "resources": {
            "html": [
                "main.html"
            ]
        },
        "title": "title-2"
    }
]

Dashboard widgets

Starting with Squirro release 2.4.5 you can also use the squirro_asset tool in place of the squirro_widget tool to upload and list custom dashboard widgets using the same options as for dashboard loaders.

To upload:

> squirro_asset widget upload --folder <dashboard-widget-folder> --token <token> --cluster <cluster-url>

and to list widgets:

> squirro_asset widget list --token 0da7dc97bdacebf6123fb61f34482f247b8eb37f5a19cba0e296851b5094e77aa5edbc96041f34ad3aba43bd4708267f1cf09b41028137cf57e53f7ae9829998 --cluster https://testing.squirro.net
[
    {
        "hash": "9a6cee78-28e5-11e6-90b4-843835563242",
        "name": "name-1",
        "title": "title-1",
        "directory": "/var/lib/squirro/topic/widgets/tenant",
        "resources": {
            "html": [
                "cells.html",
                "row.html",
                "widget.html",
                "header.html"
            ],
            "css": [
                "widget.css"
            ],
            "js": [
                "widget.js"
            ]
        }
    },
    {
        "directory": "/var/lib/squirro/topic/widgets/tenant",
        "hash": "228da928-c076-11e6-b6c3-80e65013fba6",
        "name": "name-2",
        "resources": {
        }
    }
]

Pipelets

Also with release 2.4.5 you can use the squirro_asset tool in place of the pipelet tool mentioned under Pipelets to manage pipelets. Instead of

> pipelet <any options>

you can using squirro_asset followed by pipelet as asset type like so:

> squirro_asset pipelet <any options used for squirro_pipelet>

For convenience you can use the --help option to obtain detailed help on squirro_asset pipelet commands and options:

> squirro_asset pipelet --help
usage: squirro_asset pipelet [-h]
                             {consume,list,upload,source,rerun,validate} ...

positional arguments:
  {consume,list,upload,source,rerun,validate}
                        sub-command help
    validate            Validates that the script doesn't violate any
                        restrictions
    consume             Runs one or more items through the consume method of
                        the pipelet
    rerun               Re-runs a pipelet on the server
    upload              Uploads the script to Squirro
    source              Creates a source that is configured with `name`
    list                List all available pipelets

optional arguments:
  -h, --help            show this help message and exit