Providers Connectors are the interface between an external data source and Squirro. They ensure that any new content from the source system is delivered to Squirro automatically. Providers Connectors are suited very well for connecting very large source data sets, of which only a partial selection needs to be imported into Squirro. For example, the built-in Twitter provider Twitter connector will only import tweets for users or keywords which a user has actively subscribed to from Squirro.
For regular imports of smaller data sets where no user interaction is required for data selection, the Loading Tools or the SDK tools are less complex options.
Table of Contents
| Table of Contents | ||||
|---|---|---|---|---|
|
Interaction
...
with connectors
Providers are Connectors are registered in Squirro. When adding a source, a provider a connector and configuration need to be specified (see the Create Subscription resource). The corresponding provider corresponding connector is then contacted to register the new configuration.
From that moment in time, the provider the connector will deliver any items matching the configuration to Squirro.
...
Connector Types
Most providers connectors are implemented either by regularly polling the source system or by implementing some push technology such as webhooks. For Squirro's internal providers, the type has an impact on pricing.
Polling
...
Connectors
A polling provider connectors regularly checks with the source system if new data is available. Whenever that is the case, the provider the connectors then pushes the data into Squirro.
Because polling providers polling connectors need to check the source system continually, each check may incur charges in the source system. To reduce these costs, Squirro's internal polling providers polling connectors usually implement a back-off for retrying. The poll interval is continually increased for sources where no new data is available.
Push
...
Connectors
Push providers Push connectors are notified by their source system when new data is available. For some providers some connectors these updates are near real-time.
Pricing of the source system is generally done per notification or per item.
Configuration
Provider Connector configurations are passed in as JSON dictionaries. The supported configuration keys depend on the individual provider individual connectors and are documented in the respective section.
Built-in
...
Connectors
| Include Page | ||||
|---|---|---|---|---|
|
Custom
...
Connectors
The reference documentation for how to write a custom data provider data connectors is available on the Custom Provider pageConnectors page.