To gather insights from data, Squirro needs very fast access to that data. For that purpose, any required data is indexed into the Squirro data storage layer (see the Architecture for details). Resulting sizing considerations are discussed in the Knowledge Base article How does Squirro scale for large datasets.
There are three basic ways to add data to Squirro:
- Providers connect external data sources by implementing a standardized Squirro provider web service interface. Users can add configurations to providers by adding sources in the Squirro user interface.
- The loading tools are command line executables to load data into Squirro. Both structured (CSV, Excel) as well as unstructured (e.g. PDF, Microsoft Word, …) data can be imported.
- With the SDK tools developers can integrate custom data sources without the added complexity of creating a provider.