Page tree
Skip to end of metadata
Go to start of metadata
The data loader can be extended with data loader plugins. Those plugins are used to create custom data connectors, to sources that are not yet included in Squirro.

Squirro also distributes some connectors as data loader plugins for maximum flexibility. Please see the section Data Loader Plugins in the downloads section (license required).

Table of Contents


Data loader plugins are created as a Python class (see the DataSource Class reference for how exactly).

That plugin is then loaded into the data loader using the --source-script parameter. For example:

squirro_data_load ^
    -v ^
    --cluster %CLUSTER% ^
    --project-id %PROJECT_ID% ^
    --token %TOKEN% ^
    --source-script ^
    --map-id PMID ^
    --map-title TI


The Data Loader Tutorial section Custom Data Source shows how to create a custom data loader plugin.

Class Reference

See the DataSource Class.

Custom Packages

Data loader plugins sometimes need custom Python packages. These can be provided in the pkg folder, relative to where the data loader is invoked.

The recommendation is to include a requirements.txt file in the folder where the data loader plugin is located. That file lists all the required packages, with one package dependency per line. For example:


To download these packages into the pkg folder, execute the following command (the pkg folder must exist before calling this command):

pip install -r requirements.txt --root pkg

Earlier data loaders need the following download command instead. Unfortunately that may not work in all cases, specifically if files of the type .whl are downloaded. Please contact support, if you encounter any issues in this regard.

pip install -r requirements.txt --download pkg
  • No labels