...
| Table of Contents | ||||||||
|---|---|---|---|---|---|---|---|---|
|
Overview
It is the step that The Change Pipeline step enables the creation of pipeline stages: chaining multiple pipeline workflows, where each workflow indexes the data into Squirro, and then passes them to the next pipeline workflow for further processing.
For example, it can be used as the last step of a lightweight Pipeline Workflow workflow in order to index the data as fast as possible, and then redirect them to a more heavyweight workflow which includes computationally expensive steps.
...
The picture above depicts two workflows:
the Light Workflow, which includes a small set of steps that they are generally fast to execute, and as its last step, the Change Pipeline step which is configured to redirect the data into the Heavy Workflow.
the Heavy Workflow, which includes
...
a fictional “Heavy Pipelet” step, which represents a pipelet that is computationally expensive and may take a lot of time to execute.
In this scenario, the data will undergo a basic enrichment and get indexed into Squirro by the Light Workflow, and then continue to the Heavy Workflow, which will further enrich the items and finally index them into Squirro again in order for the new enrichment to become available.
Configuration
Field | Description |
|---|---|
workflow_name | The name of the Pipeline Workflow into which the data will be redirected. |