Table of Contents
Table of Contents | ||||||
---|---|---|---|---|---|---|
|
class squirro_client.item_uploader.ItemUploader
class squirro_client.item_uploader.ItemUploader(token=None, project_id=None, project_title=None, object_id=None, source_name=None, source_ext_id=None, cluster=None, client_cls=None, batch_size=None, config_file=None, config_section=None, processing_config=None, steps_config=None, source_id=None, source_secret=None, pipeline_workflow_name=None, pipeline_workflow_id=None, timeout_secs=None, non_retry_list=[200, 202, 400, 401, 403, 404], **kwargs)
Item uploader class. Defaults are loaded from the .squirrorc
file in the current user’s home directory.
project_id
or project_title
has to be passed in.source_name
.squirro
.List of status codes for which we don’t want a retry/backoff logic. Defaults to [200, 202, 401, 403, 400, 404] 200, 202: Successful codes. 401, 403: Already have a retry block in the _perform_request method
400, 404: Does not make sense to retry for these codes as retrying
Typical usage:
>>> from squirro_client import ItemUploader >>> uploader = ItemUploader(project_title='My Project', ... token='<your token>') >>> items = [{'id': 'squirro-item1', ... 'title': 'Items arrived in Squirro!'}] >>> uploader.upload(items)
Project selection:
The ItemUploader creates a source in your project. The project must exist before the ItemUploader is instantiated.
Source selection:
The source will be created or re-used, the above parameter define how the source will be named.
Configuration:
The ItemUploader
can load its settings from a configuration file The default section is squirro
and may be overridden by the parameter config_section
to allow for multiple sources/projects.
Example configuration:
[squirro] project_id = 2sic33jZTi-ifflvQAVcfw token = 9c2d1a9002a8a152395d74880528fbe4acadc5a1
upload
upload(items, priority=0, pipeline_workflow_id=None, num_retries=10, delay=1, backoff=2)
Sends items
to Squirro.
Info |
---|
The ItemUploader Reference has now been moved to the Developer Documentation and can be accessed there. |