...
Code Block | ||||
---|---|---|---|---|
| ||||
from squirro_client import SquirroClient client = SquirroClient(None, None, cluster='https://next.squirro.net/') client.authenticate(refresh_token='293d…a13b') # get existing source configuration (including processing configuration) source = client.get_subscription(project_source(projectid='…', object_id='…', sourcesubscription_id='…') config = source.get('config', {}) processing_config = config.get('processing_config', {}) # modify processing configuration processing_config['deduplication'] = { 'enabled': True, 'policy': 'update', 'deduplication_fields': [ ['external_id'], ['link'], ['title'], ], } config['processing'] = processing_config client.modify_subscription(project_source(projectid='…', object_id='…', sourcesubscription_id='…', config=config) |
In the example above the processing pipeline is instructed to update duplicate items based on the external_id
, link
and title
properties individually.