This page will describe in detail how you can build a custom loader to work with data formats/inputs that are not supported out-of-the-box.
Table of Contents
| Table of Contents | ||||
|---|---|---|---|---|
|
Prerequisites
Follow the steps outlined here: Data Loader Tutorial#Setup.
...
See the full information on squirro_asset Command Line Reference, but in a nutshell this is how a data loader plugin can be uploaded:
| Code Block | ||
|---|---|---|
| ||
squirro_asset dataloader_plugin upload --folder pubmed --token %TOKEN% --cluster %CLUSTER% |
| Expand | ||
|---|---|---|
| ||
|
Preview
Apart from technical implementation differences between the command line and frontend data load which are not visible to the users, the main consideration for writing a UI compatible loader is the preview mode.
...
Data loader plugins often need to cache information or store certain progress information. For these purposes there are two types of stores that are available to use inside a data loader plugin:
key_value_cache
key_value_store
This is covered in Data loader API for Caching and Custom State Management.