The config file used for Pipelets has a strict JSON format. In the following chapters you can find the description of the attributes used in these files and some examples.
Table of Contents
Pipelets config file
When creating a new pipelet it is mandatory to create a config file which references the pipelet itself.
Sample file
Below is a valid JSON file used for pipelet configuration.
{ "DummyPipeletClass":{ "file_location":"pipelets\dummy_pipelet.py", "stage":"before templating", "config": { "restricted_fields_columns": ["HeaderRestrictedFieldNames", "DetailRestrictedFieldNames"] } }, "DummyPipeletClass2":{ "class_name": "DummyPipeletClass" "file_location":"pipelets\dummy_pipelet.py", "stage":"after templating", "config": { "restricted_fields_columns": ["HeaderRestrictedFieldNames", "DetailRestrictedFieldNames"] } } }
Structure
The format of the pipelets config file is Json. In the examples above, the curly brackets at the root of the file open a new dictionary. Within each dictionary there are a number of keys, each representing a pipelet class. The attributes of each class will be described below.
Reference
The pipelets config files contains the following attributes.
Key | Data Type | Description | Notes |
---|---|---|---|
config | Dictionary | Contains the attributes of each piplet. This dictionary is passed in as a constructor argument to the pipelet. | |
file_location | String | The location of the Python file. This is mandatory. It can be the full path or the relative location from where the tool is executed | |
stage | String | Specifies when the pipelets consume method will be executed. Possible values:
| |
class_name | String | The name of main python class of the pipelet. The class is expected to be a child class of PipeletV1. If class_name is missing, the key of the top-level dictionary is used to determine class name (DummyPipeletClass in the example above). | Introduced in version 2.6.5. In earlier version the top-level key was always used. Specifying this attribute allows running the same pipelet multiple times, each with a different config |