| Excerpt |
|---|
The squirro_asset tool can be used to manage all Squirro code artefacts called custom assets. Assets are Pipelets, Custom Widgets, and Custom Dashboard Loaders. |
Table of Contents
Starting with the 2.4.5 Squirro release, you can use the squirro_asset tools to manage all the custom "code artifacts" in Squirro server systems such as Dashboard Loaders, Dashboard Widgets, or Pipelets.
The first parameter is the type of asset: dashboard_loader, widget, or pipelet, and if at any time you are unsure of what parameters are allowed, you can request help by adding the --help parameter.
| Code Block |
|---|
|
> squirro_asset --help
usage: squirro_asset [-h] [--verbose] [--log-file LOG_FILE] [--version]
{widget,pipelet,dashboard_loader,dataloader_plugin} ...
Squirro asset management tool
positional arguments:
{widget,pipelet,dashboard_loader}
asset type help
widget Manage custom widgets
pipelet Manage custom pipelets
dashboard_loader Manage custom dashboard_loader
dataloader_plugin Manage custom dataloader_plugin
General Options:
-h, --help Show this help message and exit.
--verbose, -v Show additional information.
--log-file LOG_FILE Log file on disk.
--version show program's version number and exit |
Dataloader plugin
| Warning |
|---|
Note that the dataloader plugins using the deprecated method `add_args` can not be used with the self service Squirro dataloader UI. Please migrate the `add_args` method to `getArguments` method as listed on this page |
You can upload new or modified dataloader plugins using the upload command, for example:
| Code Block |
|---|
|
> squirro_asset dataloader_plugin upload --folder <dataloader-plugin-folder> --token <token> --cluster <cluster-url> |
where <dataloader-plugin-folder> is a directory containing the following files:
- Dataloader plugin python file
- A `requirements.txt` listing any external python dependencies needed for the plugin
- A png file to be used as a logo for the dataloader plugin in the Squirro UI
- A `dataloader_plugin.json` file containing the proper metadata for the dataloader plugin (see format below) in hjson format
Format of `dataloader_plugin.json` file
| Code Block |
|---|
{
# title of the dataloader plugin to be displayed in the UI
"title": "Hipchat",
# descriptuon of the dataloader plugin to be displayed in the UI
"description": "For importing data into hipchat",
# Name of the plugin file
"plugin_file": "hipchat.py",
# Name of the requirements file containing the python dependencies
"requirements_file": "requirements.txt",
# Category under which to show this dataloader plugin, Possible options are:
# 'Web', 'Social Media', 'Data Import', 'Developers', 'Enterprise', 'Business Intelligence', 'CRM', 'ITSM'
"category": "developers",
# Name of the PNG file to be used as a thumbnail while displaying the dataloader plugin in the UI
"thumbnail_file": "Hipchat_Atlassian_logo.png"
} |
You can list all dataloader plugins using the list option
| Code Block |
|---|
|
> squirro_asset dataloader_plugin list --token <token> --cluster <cluster-url>
[
{
"requirements_file": "requirements.txt",
"category": "developers",
"config_options": [
{
"required": true,
"name": "hipchat_url",
"help": "URL of HipChat public web interface."
},
{
"name": "hipchat_endpoint",
"help": "Endpoint of the HipChat API (URL)"
},
{
"required": true,
"name": "hipchat_authtoken",
"help": "Authentication token for HipChat API"
},
{
"default": 3,
"type": "int",
"name": "hipchat_conversation_length",
"help": "Number of days back into the past for which to fetch the message archive counting back from yesterday"
},
{
"default": 1,
"type": "int",
"name": "hipchat_conversation_age",
"help": "Number of days back into the past for which to fetch the message archive counting back from yesterday"
},
{
"name": "hipchat_room_blacklist",
"help": "Space-separated list of rooms to ignore"
},
{
"default": "message",
"name": "hipchat_mode",
"help": "Whether to individual messages (realtime) or groups conversations (older than 1 day)"
}
],
"hash": "d8c6ffe1-1886-11e7-a5a4-f45c89acc4cb",
"description": "For importing data into hipchat",
"title": "Hipchat",
"pipelets_files": [
"keywords_pipelet1.py",
"sentiment_pipelet2.py"
],
"thumbnail_file": "Hipchat_Atlassian_logo.png",
"name": "hipchat_thumbnail",
"body_template_file": "body.j2",
"thumbnail_b64_encoded": "+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+",
"plugin_file": "hipchat.py",
"resources": {
"txt": [
"requirements.txt"
],
"py": [
"keywords_pipelet.py",
"hipchat.py",
"load.py"
],
"pyc": [
"hipchat.pyc"
],
"png": [
"Hipchat_Atlassian_logo.png"
]
},
"title_template_file": "title.j2"
}
] |
Dashboard loaders
You can upload new or modified dashboard loaders using the upload command, for example:
| Code Block |
|---|
|
> squirro_asset dashboard_loader upload --folder <dashboard-loader-folder> --token <token> --cluster <cluster-url> |
where <dashboard-folder> is a directory containing dashboard loader html, css and javascript files along with a file named dashboard_loader.json formatted in hjson, and you specify one of the options to connect to a Squirro server installation (using <token> and <cluster-url> in the above example).
You can list all dashboard loaders using the same parameters as for upload except --folder:
| Code Block |
|---|
|
> squirro_asset dashboard_loader upload --token <token> --cluster <cluster-url>
[
{
"hash": "ec43609e-c5c6-11e6-ac8a-80e65013fba6",
"name": "name-1",
"title": "title-1",
"directory": "/var/lib/squirro/topic/assets/dashboard_loader/tenant",
"resources": {
"css": [
"mycss.css"
]
}
},
{
"directory": "/var/lib/squirro/topic/assets/dashboard_loader/tenant",
"hash": "bed8e082-c395-11e6-94ff-000c29dfbc49",
"name": "name-2",
"resources": {
"html": [
"main.html"
]
},
"title": "title-2"
}
] |
Starting with Squirro release 2.4.5 you can also use the squirro_asset tool in place of the squirro_widget tool to upload and list custom dashboard widgets using the same options as for dashboard loaders.
To upload:
| Code Block |
|---|
> squirro_asset widget upload --folder <dashboard-widget-folder> --token <token> --cluster <cluster-url> |
and to list widgets:
| Code Block |
|---|
> squirro_asset widget list --token 0da7dc97bdacebf6123fb61f34482f247b8eb37f5a19cba0e296851b5094e77aa5edbc96041f34ad3aba43bd4708267f1cf09b41028137cf57e53f7ae9829998 --cluster https://testing.squirro.net
[
{
"hash": "9a6cee78-28e5-11e6-90b4-843835563242",
"name": "name-1",
"title": "title-1",
"directory": "/var/lib/squirro/topic/widgets/tenant",
"resources": {
"html": [
"cells.html",
"row.html",
"widget.html",
"header.html"
],
"css": [
"widget.css"
],
"js": [
"widget.js"
]
}
},
{
"directory": "/var/lib/squirro/topic/widgets/tenant",
"hash": "228da928-c076-11e6-b6c3-80e65013fba6",
"name": "name-2",
"resources": {
}
}
] |
Pipelets
Also with release 2.4.5 you can use the squirro_asset tool in place of the pipelet tool mentioned under Pipelets to manage pipelets. Instead of
| Code Block |
|---|
> squirro_pipelet <any options> |
you can using squirro_asset followed by pipelet as asset type like so:
| Code Block |
|---|
> squirro_asset <any options used for squirro_pipelet> |
For convenience you can use the --help option to obtain detailed help on squirro_asset pipelet commands and options:
| Code Block |
|---|
> squirro_asset pipelet --help
usage: squirro_asset pipelet [-h]
{consume,list,upload,source,rerun,validate} ...
positional arguments:
{consume,list,upload,source,rerun,validate}
sub-command help
validate Validates that the script doesn't violate any
restrictions
consume Runs one or more items through the consume method of
the pipelet
rerun Re-runs a pipelet on the server
upload Uploads the script to Squirro
source Creates a source that is configured with `name`
list List all available pipelets
optional arguments:
-h, --help show this help message and exit |