The Ingester service offers a set of configuration options that control how it handles the priorities set on the data batches that derive from the configured data sources. For information on how you can define the priorities for your data sources, please refer to the Priorities for Data Sources page.
To access those configuration options, go to the Server space and then select Configuration. You can reveal those options by writing ingester in the search bar.
...
A description of these configuration options is listed in the following table.
...
Name
...
Default
...
Description
...
ingester.priorities.enabled
...
true
...
Feature flag to enable or disable handling of batches with priority levels.
By default, it is enabled, which means that the Ingester service understands priorities.
...
ingester.priorities.optimize-processors-utilization
...
false
...
Flag to enable or disable the optimization of allowing processors of lower priorities to consume batches of higher priorities.
For example, when this optimization is enabled (true), the processors in the low and normal pools, in addition to the batches of their designated priority, will also consume batches of high priority.
By default, this optimization is disabled.
...
ingester.priorities.pool-high-processors
...
1
...
Define the number of processors in the pool which consumes high priority batches.
...
ingester.priorities.pool-normal-processors
...
1
...
Define the number of processors in the pool which consumes normal priority batches.
...
ingester.priorities.pool-low-processors
...
1
...
Define the Number of processors in the pool which consumes low priority batches.
...
This page can now be found at Pipeline Prioritization on the Squirro Docs site.