ingester.ini config file, located at /etc/squirro/ingester.ini configures the workings of the Ingester ServiceTable of Contents
Example Configuration
[ingester] # number of new pipeline processors processors = 5 # if there are no items to process, how long to wait until checking again sleep_in_secs_if_no_batch = 5 # how often the ingester file reaper should should check for orphaned files in content streamer clean_up_interval_in_hours = 1 [processor] # number of workers to use on non-batched steps workers = 10 # maximum number of retries to retry an item that failed processing a step max_retries = 10
Data Retention Configuration
Data retention is configured in common.ini using the following default options:
[content_filesystem_stream] # root directories for ingester content stream (space separated if more than one) data_directories = /var/lib/squirro/inputstream # number of days and hours we keep around item batches that failed to be ingested # total time is days + hours days_to_retain_failed_batches = 30 hours_to_retain_failed_batches = 0