Squirro also distributes some connectors as data loader plugins for maximum flexibility. Some data loader plugins can also be installed from Squirro's repository. See Plugin Repository for more.
Usage
Data loader plugins can be used in Squirro's user interface, or on the command line with the Data Loader.
When using a data loader plugin in the user interface, it behaves exactly the same way as the built-in connectors. You can configure some options, select the mappings, and save the source.
On the command line, the plugin can be loaded by specifying the --source-script
option. For example:
squirro_data_load ^ -v ^ --cluster %CLUSTER% ^ --project-id %PROJECT_ID% ^ --token %TOKEN% ^ --source-script medline.py ^ --map-id PMID ^ --map-title TI
As can be seen, --source-script
is used in place of the --source-type
option. Apart from that, the behaviour is identical, and mapping, facet configuration, etc. all behave identically.
Writing plugins
Data loader plugins are implemented as Python classes. The topic is covered in Writing a custom data loader.