Data Loader Facet Config Reference

The Data Loader uses config files for facet creation, which has an HJson format (which is a more human friendly superset of Json). In the following chapters you can find the description of the attributes used in these files and some examples.

Table of Contents

Facets config file

Sample file

Below is an example of a fully valid Hjson file which is not a valid Json. Notice how some of the keys/values are missing quotes and some rows are missing commas a the the end. 

{
	"InteractionSubject": {
		"name": "interaction_subject",
		"display_name": "Interaction Subject",
		"group_name": "Interactions",
		"visible": true,
		"searchable":true,
        "typeahead":true
	},
	"InteractionType": {
		"name": "interaction_type",
		"display_name": "Interaction Type",
		"group_name": "Interactions",
		"visible": true,
		"searchable":true,
        "typeahead":false
	},
	"Date": {
        "name": "creation_date",
        "display_name": "Date of Creation",
		"data_type": "datetime",
		"input_format_string": "%Y-%m-%dT%H:%M:%S",
		"default_value":"1900/01/01",
		"visible": true
	},
	"EntitledUsers": {
        "name": "users",
        "display_name": "Users",
        "auth": true,
        "auth_value_mandatory": true,
        "import": false
    },
    "InternalAttendees": {
        "name": "attendees",
        "display_name": "Attendees",
        "group_name": "Interactions",
        "pivotal_group": "Members",
        "delimiter": ";",
        "visible": true
    },
	"NoAtendees": {
        "name": "number_of_attendees",
        "display_name": "Number of Attendees",
		"data_type": "int",
		"visible": true,
		"searchable":false
	},
}

Best Practices of Facet Naming

Follow the best practices for facet names and have a user friendly, nice looking name in the "display_name" but mind the maximum length mentioned below.

Best practices for the facet names:

  • Only use lower case
  • Use an underscore as a separator to join multiple words
  • Avoid using space as a separator
  • Avoid having any special characters in the facet name

Structure: Facet Keys and Facet Attributes

In the examples above, the curly brackets at the root of the file open a new dictionary. Within each dictionary there are a number of facet keys which, in turn have their own dictionary of key / value pairs - the facet attributes.

  • Facet Keys are used to identify data elements in the data source the facets are being applied to, for example the column headings in an Excel spreadsheet.
  • Facet Attributes are used to define the behaviour of the facet and how it shold be handled by Squirro. See the reference below for more details.


Character Encoding and Facet Keys

Be careful using characters outside of the printable ASCII encoding range, such as umlauts (ü) or diacritics (é) in the facet key above. JSON files are normally encoded as UTF-8 while Excel, for example, will default to using ANSI encoding such as Windows-1252. That means if you use a value such as "Zürich" in a facet key in your JSON, it will only match the corresponding column in the input data if that data is also encoded as UTF-8. Recent versions of Excel 2016 can now export data to "CVS UTF-8" but if in doubt, simple restrict column names to characters in the printable ASCII range (i.e. avoid umlauts).


Facet Attributes Reference

Each facet in the the facets config file may be configured with the following facet attributes.

KeyData TypeDefaultDescription
analyzedBooleantrueOnly applies to data_type string. If set to False the field is not analyzed for searching. It can be used only for display and filtering & aggregation. This speeds up index operations and keeps the index size smaller.

auth

Boolean

false

If set to 'true' for a field, this field will be used to determine user access to the Squirro item.

auth_value_mandatory

Boolean


If value of the field in the data source is empty, the item will not be uploaded.

data_type

String

'string'

Data type of the facet. One of ('string', 'int', 'float', 'datetime','geo_point', 'weighted').

default_value

String

None

Default value of the facet if value is None.

delimiter

String


Specified for columns that have multiple values, i.e. pivotal and auth columns.

display_name

String


Name to show to the user in the front-end. Maximum 50 characters

format_instr

String


Formatting instruction for the facet value display. This parameter defines how the Squirro UI will display the value. This is only relevant for int, float and dates. It’s also documented here.

group_name

String


Label of the group this facet is under. Maximum 50 characters.

import

Boolean

true

For specifying if a column will be loaded as a facet or not.

input_format_string

String or List of strings


Format of the date string coming from the source (used for csv and excel). When importing from database, the date values are of type datetime and don’t need a format. Only for datetime columns.

Can be specified as a list of format strings. The data loader then tries each format string in order and uses the first one that doesn't result in a parsing error.

name

String


Name use to display and access the facet in the Squirro UI - used to identify facets in the search field for example and on item tags. Maximum 50 characters

pivotal_group

String


If this attribute exists then the column is also a pivotal column. That means that the values of that column, delimited by 'delimiter', were joined from a detail table to generate only one line in the source. You can find more information here.

searchable

Boolean

false

If 'true' this facet values will be searchable.

typeaheadBooleanfalseIf 'true', this facet will be included in the typeahead search function of the various search fields. Added with version 2.4.0 (Aspen)

visible

Boolean

true

If `false` this facet will be hidden in the front-end.