The squirro_asset
tool can be used to manage all Squirro code artefacts called custom assets. Assets are Pipelets, Custom Widgets, and Custom Dashboard Loaders.
Table of Contents
squirro_assets tool
Starting with the 2.4.5 Squirro release, you can use the squirro_asset
tools to manage all the custom "code artifacts" in Squirro server systems such as Dashboard Loaders, Dashboard Widgets, or Pipelets.
The first parameter is the type of asset: dashboard_loader
, widget
, or pipelet
, and if at any time you are unsure of what parameters are allowed, you can request help by adding the --help
parameter.
> squirro_asset --help usage: squirro_asset [-h] [--verbose] [--log-file LOG_FILE] [--version] {widget,pipelet,dashboard_loader,dataloader_plugin} ... Squirro asset management tool positional arguments: {widget,pipelet,dashboard_loader} asset type help widget Manage custom widgets pipelet Manage custom pipelets dashboard_loader Manage custom dashboard_loader dataloader_plugin Manage custom dataloader_plugin General Options: -h, --help Show this help message and exit. --verbose, -v Show additional information. --log-file LOG_FILE Log file on disk. --version show program's version number and exit
Dataloader plugin
Note that the dataloader plugins using the deprecated method `add_args` can not be used with the self service Squirro dataloader UI. Please migrate the `add_args` method to `getArguments` method as listed on this page
You can upload new or modified dataloader plugins using the upload command, for example:
> squirro_asset dataloader_plugin upload --folder <dataloader-plugin-folder> --token <token> --cluster <cluster-url>
where <dataloader-plugin-folder>
is a directory containing the following files:
- Dataloader plugin python file
- A `requirements.txt` listing any external python dependencies needed for the plugin
- A png file to be used as a logo for the dataloader plugin in the Squirro UI
- A `dataloader_plugin.json` file containing the proper metadata for the dataloader plugin (see format below) in hjson format
Format of `dataloader_plugin.json` file
{ # title of the dataloader plugin to be displayed in the UI "title": "Hipchat", # descriptuon of the dataloader plugin to be displayed in the UI "description": "For importing data into hipchat", # Name of the plugin file "plugin_file": "hipchat.py", # Name of the requirements file containing the python dependencies "requirements_file": "requirements.txt", # Category under which to show this dataloader plugin, Possible options are: # 'Web', 'Social Media', 'Data Import', 'Developers', 'Enterprise', 'Business Intelligence', 'CRM', 'ITSM' "category": "developers", # Name of the PNG file to be used as a thumbnail while displaying the dataloader plugin in the UI "thumbnail_file": "Hipchat_Atlassian_logo.png" }
You can list all dataloader plugins using the list option
> squirro_asset dataloader_plugin list --token <token> --cluster <cluster-url> [ { "requirements_file": "requirements.txt", "category": "developers", "config_options": [ { "required": true, "name": "hipchat_url", "help": "URL of HipChat public web interface." }, { "name": "hipchat_endpoint", "help": "Endpoint of the HipChat API (URL)" }, { "required": true, "name": "hipchat_authtoken", "help": "Authentication token for HipChat API" }, { "default": 3, "type": "int", "name": "hipchat_conversation_length", "help": "Number of days back into the past for which to fetch the message archive counting back from yesterday" }, { "default": 1, "type": "int", "name": "hipchat_conversation_age", "help": "Number of days back into the past for which to fetch the message archive counting back from yesterday" }, { "name": "hipchat_room_blacklist", "help": "Space-separated list of rooms to ignore" }, { "default": "message", "name": "hipchat_mode", "help": "Whether to individual messages (realtime) or groups conversations (older than 1 day)" } ], "hash": "d8c6ffe1-1886-11e7-a5a4-f45c89acc4cb", "description": "For importing data into hipchat", "title": "Hipchat", "pipelets_files": [ "keywords_pipelet1.py", "sentiment_pipelet2.py" ], "thumbnail_file": "Hipchat_Atlassian_logo.png", "name": "hipchat_thumbnail", "body_template_file": "body.j2", "thumbnail_b64_encoded": "+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+", "plugin_file": "hipchat.py", "resources": { "txt": [ "requirements.txt" ], "py": [ "keywords_pipelet.py", "hipchat.py", "load.py" ], "pyc": [ "hipchat.pyc" ], "png": [ "Hipchat_Atlassian_logo.png" ] }, "title_template_file": "title.j2" } ]
Dashboard loaders
You can upload new or modified dashboard loaders using the upload command, for example:
> squirro_asset dashboard_loader upload --folder <dashboard-loader-folder> --token <token> --cluster <cluster-url>
where <dashboard-folder>
is a directory containing dashboard loader html, css and javascript files along with a file named dashboard_loader.json
formatted in hjson, and you specify one of the options to connect to a Squirro server installation (using <token> and <cluster-url> in the above example).
You can list all dashboard loaders using the same parameters as for upload except --folder
:
> squirro_asset dashboard_loader upload --token <token> --cluster <cluster-url> [ { "hash": "ec43609e-c5c6-11e6-ac8a-80e65013fba6", "name": "name-1", "title": "title-1", "directory": "/var/lib/squirro/topic/assets/dashboard_loader/tenant", "resources": { "css": [ "mycss.css" ] } }, { "directory": "/var/lib/squirro/topic/assets/dashboard_loader/tenant", "hash": "bed8e082-c395-11e6-94ff-000c29dfbc49", "name": "name-2", "resources": { "html": [ "main.html" ] }, "title": "title-2" } ]
Dashboard widgets
Starting with Squirro release 2.4.5 you can also use the squirro_asset
tool in place of the squirro_widget tool to upload and list custom dashboard widgets using the same options as for dashboard loaders.
To upload:
> squirro_asset widget upload --folder <dashboard-widget-folder> --token <token> --cluster <cluster-url>
and to list widgets:
> squirro_asset widget list --token 0da7dc97bdacebf6123fb61f34482f247b8eb37f5a19cba0e296851b5094e77aa5edbc96041f34ad3aba43bd4708267f1cf09b41028137cf57e53f7ae9829998 --cluster https://testing.squirro.net [ { "hash": "9a6cee78-28e5-11e6-90b4-843835563242", "name": "name-1", "title": "title-1", "directory": "/var/lib/squirro/topic/widgets/tenant", "resources": { "html": [ "cells.html", "row.html", "widget.html", "header.html" ], "css": [ "widget.css" ], "js": [ "widget.js" ] } }, { "directory": "/var/lib/squirro/topic/widgets/tenant", "hash": "228da928-c076-11e6-b6c3-80e65013fba6", "name": "name-2", "resources": { } } ]
Pipelets
Also with release 2.4.5 you can use the squirro_asset
tool in place of the pipelet tool mentioned under Pipelets to manage pipelets. Instead of
> squirro_pipelet <any options>
you can using squirro_asset followed by pipelet as asset type like so:
> squirro_asset <any options used for squirro_pipelet>
For convenience you can use the --help option to obtain detailed help on squirro_asset pipelet commands and options:
> squirro_asset pipelet --help usage: squirro_asset pipelet [-h] {consume,list,upload,source,rerun,validate} ... positional arguments: {consume,list,upload,source,rerun,validate} sub-command help validate Validates that the script doesn't violate any restrictions consume Runs one or more items through the consume method of the pipelet rerun Re-runs a pipelet on the server upload Uploads the script to Squirro source Creates a source that is configured with `name` list List all available pipelets optional arguments: -h, --help show this help message and exit