Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Add example of discarding items by returning None from consume

...

As it name says it does nothing but return the item unchanged.

Modifying Items

The item is a Python dict type and can be modified before it is returned. The available item fields are documented in the Item Format reference. The following example illustrates modifying an item:

...

This pipelet will modify each item it processes, appending the string "Hello, World!" to the title.

Skipping Items

Items can be "skipped" ( i.e. not added to Squirro's search index ) by return None instead of the item for example;

Code Block
languagepy
from squirro.sdk import PipeletV1
 
class SkipItemPipelet(PipeletV1):
    def consume(self, item):
        if not item.get('title', '').startswith('[IMPORTANT]')
            return None
        return item

In this example we discard all items where the title does not start with the string "[IMPORTANT]".

Returning multiple items

The pipelet is always called for each item individually. But in some use cases the pipelet should not just return one item but multiple ones. In those cases use the Python yield statement to return each individual item. For example:

...