Versions Compared

Old Version 22

changes.mady.by.user Chiara Sannitz

Saved on

compared with

New Version Current

changes.mady.by.user Chiara Sannitz

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

You can configure Squirro to collect data on the user's interactions with Squirro. The collected activity data provides insights on the overall usage of the system, for example, what dashboards were visited or what queries a user submitted. Squirro uses the recorded queries to improve the overall user experience by suggesting popular queries in typeahead or recommending similar queries when a user types into the search bar.

Info

Note: If you’re looking for information about activity tracking within a Squirro Monitoring dashboard, see Squirro Monitoring.

Configuration

Enable activity tracking in the Server Configuration (frontend.userapp.log-activity).

...

The activity logs are by default stored under /var/log/squirro/frontend/$hostname/activity.$year-$month-$day.jsonl.

The path can be changed in the common.ini file by adjusting the path config in the activity section:

Code Block
[activity]
path = /var/log/squirro/frontend/

Note: the $hostname folder is always appended to the path that is specified.

Activity Tracking Project

Info

Starting with version 3.4.5, Squirro allows for automatic creation of a Squirro Monitoring project.

The Squirro Monitoring project uses the Squirro Activity Log data loader plugin with a pre-configured Activity Log Enrichment pipeline to continuously load the activity data from the activity logs.

Automatic Creation

Self-service instances are shipped with a pre-configured monitoring project. On new deployments, Squirro creates the monitoring project named Squirro Monitoring upon first admin-user sign up (for each tenant).

You can configure the behaviour of the automatic monitoring project creation in the Server space under Configuration by setting the value of topic.monitoring.default-monitoring-project to true or false (default is true)

Any automatically created monitoring comes with the following:

  • A pre-configured Squirro Activity Log data source

  • A custom index for the activity data with name squirro_v9_activity_tracking_${tenant}. The index is required to set up the popular queries suggestions.

Note

Current limitations:

  • The monitoring project has no pre-configured dashboards. Have a look in the delivery repository for a template.

Manual Creation

On existing deployments, you can also manually create a monitoring project:

  1. Create a new project

  2. Add the Squirro Activity Log data loader plugin as data source

  3. Click the SAVE button to set up the source as 1-click connector.

By setting up the source as 1-click connector, all necessary facets are automatically mapped correctly. If you need to map the facets manually, click the NEXT button (step 3 above). The relevant facets are listed in the facets.json below.

Expand
titleSquirro Activity Log: facets.json
Code Block
languagejson
{
    "source_type": {
        "name": "source_type",
        "display_name": "Source Type",
        "default_value": "Activity",
        "visible": true,
        "searchable": true,
        "typeahead": true,
        "analyzed": true
    },
    "hostname": {
        "name": "hostname",
        "display_name": "Hostname",
        "analyzed": false,
        "group_name": "Activity"
    },
    "session_id": {
        "name": "session_id",
        "display_name": "Session",
        "analyzed": false,
        "group_name": "Activity"
    },
    "pageview_id": {
        "name": "pageview_id",
        "display_name": "Pageview",
        "analyzed": false,
        "group_name": "Activity"
    },
    "source": {
        "name": "event_source",
        "display_name": "Source",
        "analyzed": false,
        "group_name": "Activity"
    },
    "action": {
        "name": "action",
        "display_name": "Action",
        "analyzed": false,
        "group_name": "Activity"
    },
    "user_id": {
        "name": "user_id",
        "display_name": "User ID",
        "analyzed": false,
        "group_name": "User"
    },
    "query.query": {
        "name": "query",
        "display_name": "Query",
        "group_name": "Activity",
        "searchable": true
    },
    "query_context.parse.user_terms": {
        "name": "query_user_terms",
        "display_name": "Query User Terms",
        "group_name": "Activity",
        "analyzed": false
    },
    "query_context.parse.facet_filters": {
        "name": "query_user_filters",
        "display_name": "Query User Filters",
        "group_name": "Activity",
        "analyzed": false
    },
    "query_context.parse.language": {
        "name": "query_language",
        "display_name": "Query Language",
        "group_name": "Activity",
        "analyzed": false
    },
    "query_context.parse.type": {
        "name": "query_type",
        "display_name": "Query Type",
        "group_name": "Activity",
        "analyzed": false
    },
    "item.id": {
        "name": "item_id",
        "display_name": "Item ID",
        "analyzed": false,
        "group_name": "Activity"
    },
    "item.link": {
        "name": "item_link",
        "display_name": "Item Link",
        "analyzed": false,
        "group_name": "Activity"
    },
    "drop_to": {
        "name": "drop_to",
        "display_name": "Item Dropped To",
        "analyzed": false,
        "group_name": "Activity"
    },
    "widget.type": {
        "name": "widget_type",
        "display_name": "Widget Type",
        "analyzed": false,
        "group_name": "Activity"
    },
    "dashboard.title": {
        "name": "dashboard_title",
        "display_name": "Dashboard",
        "analyzed": false,
        "group_name": "Activity"
    },
    "layer_title": {
        "name": "layer_title",
        "display_name": "Layer",
        "analyzed": false,
        "group_name": "Activity"
    },
    "dashboard.id": {
        "name": "dashboard_id",
        "display_name": "Dashboard ID",
        "analyzed": false,
        "group_name": "Activity"
    },
    "project.title": {
        "name": "project_title",
        "display_name": "Project",
        "analyzed": false,
        "group_name": "Activity"
    },
    "project.id": {
        "name": "project_id",
        "display_name": "Project ID",
        "analyzed": false,
        "group_name": "Activity"
    },
    "edit": {
        "name": "edit",
        "display_name": "Edit",
        "analyzed": false,
        "group_name": "Activity"
    },
    "microseconds": {
        "name": "microseconds",
        "display_name": "Microseconds",
        "data_type": "int",
        "group_name": "Activity"
    },
    "result.start": {
        "name": "result_start",
        "display_name": "Result List Offset",
        "data_type": "int",
        "group_name": "Activity"
    },
    "result.total": {
        "name": "result_total",
        "display_name": "Results",
        "data_type": "int",
        "group_name": "Activity"
    },
    "type": {
        "name": "type",
        "display_name": "Type",
        "group_name": "Activity"
    }
}

For manually created monitoring projects, the Squirro Activity Log data loader plugin uses a custom index with name squirro_v9_${project_id}. The index is required to set up the popular queries suggestions.

Activity Dataloader Plugin

The data loader plugin reads by default the path with activity files from the common.ini config.
If you would like to provide a custom path where your activity files are stored, you can specify the Activity path option in the advanced options.

...

Running data loader on multi-node instances and storing files in the EFS

The Squirro Activity Log data loader plugin works both on multi-node instances and with activity files placed in the EFS. It takes care of loading proper data by resolving the hostname folder where the activity files are stored.

The plugin gathers the activity files from the root level of the activity path and then check for files inside the folder with corresponding hostname. If it finds a folder with the same hostname as the hostname that it is running on, it also searches for files inside this folder.

So assuming the data loader runs on the instance with the hostname: hostname-1, it will gather the following files:

  • /var/log/squirro/frontend/activity.2022-01-01.jsonl

  • /var/log/squirro/frontend/hostname-1/activity.2022-01-02.jsonl

But won't gather the file below:

  • /var/log/squirro/frontend/hostname-2/activity.2022-01-03.jsonl

Session tracking

Info

Starting with version 3.6.5 the activities include session ID.

To get more insights of how users interact and use the Squirro product, the session ID is attached to each activity entry. Session is stored in the cookies and has the expiration time that is used for defining when the session starts and ends. Each user’s activity extends the session time. After no user’s activity within the specified time, it is assumed to be end of the session and for the further activities a new session starts.

To configure the session expiration time, you can use the configuration service:

...

This page can now be found atActivity Tracking on the Squirro Docs site.