...
| Excerpt |
|---|
The configuration of built-in enrichments is done with the processing config. This can be used both for enabling and disabling of enrichments, as well as adding additional configuration for a step. In addition you can use processing config to mark a source or project to use the new Squirro Pipeline 2.0 |
There are two places this configuration can be specified:
...
| Code Block | ||
|---|---|---|
| ||
client = SquirroClient(None, None, cluster='https://nextdemo.squirro.net/') client.authenticate(refresh_token='293d…a13b') client.new_subscription(project_id, object_id='default', provider='twitter', processing_config={ 'query': 'Squirro', 'processing': { 'deduplication': { 'enabled': False } } }) |
The enabled property is available for every built-in enrichment and can be set to true or false. Some of the enrichments have additional configuration options, that are described on the corresponding page.
To mark a source for processing by the new Squirro Pipeline 2.0:
| Code Block | ||
|---|---|---|
| ||
client = SquirroClient(None, None, cluster='https://demo-25.squirro.net/')
client.authenticate(refresh_token='293d…a13b')
client.new_subscription(project_id, object_id='default', provider='bulk',
config={
'pipeline': 'ingester',
'name': 'large_subscription_1',
'ext_id': 'large_subscription_1_id'
}) |
Project processing config
...