Items

Owned by Patrice Neff
Last updated: Apr 30, 2018 by Felix Hürlimann (Deactivated)
10 min read


Search items from a Squirro project.Items are added to projects through subscriptions.


Methods

Query items

GET/POST https://squirro-endpoint/api/topic/v0/(string: tenant)/projects/(id: project_id)/items/query

Returns a selection of items from the project. The items be filtered with a query and other parameters.

Parameters:
  • tenant – User tenant.
  • project_id – Project identifier.
JSON Data:

Data is passed in as a JSON object.

Valid object keys:

      • start – the index of the first item to be returned, useful for pagination (default: 0). Since the returned items are filtered after pagination, the start returned in the next_params field should be passed in to avoid receiving duplicate items.
      • count – the number of items returned (default 15). Since the returned items are filtered after pagination, the number of returned items may be lower than this number.
      • created_before – Date and Time value which limits the returned items to items that have been created before this date.
      • created_after – Date and Time value which limits the returned items to items that have been created after this date.
      • options – Dictionary of options that influence the result-set. Options supported are listed below
        • fold_near_duplicates- to fold near-duplicates together and filter them out of the result-stream.
        • abstract_size-to set the length of the returned abstract in number of characters.
        • update_cache- Boolean, if `False` the result won't be cached. Used for non-interactive queries that iterate over a large number of items. Defaults to `True`.
        • template_params- A dict of key/value pairs that can be used for query transformation in the jinja template.
        • query_syntax_strict- Boolean, if `False` strict syntax checking for input queries is disabled. If `True`, invalid queries will return an error. Defaults to `True`
      • query – Search for items that match this query. For each item a score is calculated which reflects how well it matches this query.
      • highlight – Dictionary containing highlight information. Keys are:
        • query– Boolean, if True the response will contain highlight information. 
        • smartfilters– List of SmartFilter names to be highlighted.
      • fields – Comma-separated list of fields to return. If this is specified, the return value is limited to only these fields. See the Item Format reference for a list of available fields.
      • aggregations – Dictionary of aggregations to execute on matching items. See Aggregation reference.
      • next_params – For pagination, this should be the unmodified dictionary returned by the previous request
      • encrypted_query – An encrypted version of query and other query params as returned by `Encrypted Query` resource. If this parameter is specified, the values of the following parameters will be ignored.
        • query, aggregations, created_before, created_after, options, fields

{
	"query": "Hitachi",
    "start": 0,
    "highlight": {
        "query": true
    }
} 
Headers:See Common Headers.
Status Codes:
  • 200 – Results returned.

See also Common Status Codes.

Returns:

Results including some pagination information.

Response fields:

  • items – List of items.
  • count– The number of items returned with this result set.
  • total – How many items match the current query.
  • eof – True if the there are no further results available to paginate to.
  • next_params – Query parameters that should be passed in with the next request when paginating.
  • aggregations – Result of item aggregation. See Aggregations reference.


{
    "count": 1,
    "now": "2014-12-17T14:27:47",
    "next_params": {
        "start": 1,
        "expected_num_results": 92
    },
    "eof": false,
    "items": [
        {
            "body": "<html><body>
<p class=\"tweet\">ABB and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system…",
            "language": "en",
            "title": "Supreme & Co.",
            "read": false,
            "created_at": "2014-12-17T13:10:45",
            "modified_at": "2014-12-17T13:11:16",
            "related_items": [],
            "id": "Fd2yF5BvQX2LlCdYPZ-Zow",
            "sources": [
                {
                    "link": "https://twitter.com/search?q=abb -is:retweet&f=realtime",
                    "title": "Twitter search for: \"abb\"",
                    "object_ids": [
                        "w--MRsdxR26LlymevPTpMA"
                    ],
                    "id": "nN3pr5BwQJevxbSzMB9Www",
                    "provider": "twitter"
                }
            ],
            "score": null,
            "link": "https://www.facebook.com/pages/Supreme-Co/356890661079035?ref=hl",
            "objects": [
                {
                    "source_ids": [
                        "nN3pr5BwQJevxbSzMB9Www"
                    ],
                    "project_id": "D9omq-WVQ0O2hxYp9fKUlA",
                    "type": "default",
                    "id": "w--MRsdxR26LlymevPTpMA",
                    "title": "default"
                }
            ],
            "keywords": {
                "from": [
                    "supremeco"
                ],
                "Sentiment": [
                    "Neutral"
                ],
                "Geo_State": [
                    "Japan"
                ],
                "verb": [
                    "post"
                ],
                "Organization": [
                    "ABB Group",
                    "Hitachi"
                ]
            },
            "starred": false,
            "external_id": "545204453427531777",
            "abstract": [
                "ABB and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system solutions in Japan.\ntinyurl.com/nzhqzrfABB",
                " and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system solutions in Japan.\nBusiness Standard reported that ABB",
                " and <squirro:highlight>Hitachi</squirro:highlight> entered into a JV for providing high voltage direct current system solutions in Japan",
                "-sales services related to the DC system of HVDC projects. <squirro:highlight>Hitachi</squirro:highlight> and ABB will take equity..."
            ]
        }
    ],
    "aggregations": {},
    "total": 92,
    "fingerprint_matches": {}
} 

Encrypted Query

GET/POST https://squirro-endpoint/api/topic/v0/(string: tenant)/projects/(id: project_id)/items/query_encryption

Returns the encrypted and signed version of the query and other query parameters. This encrypted query can then be used with the `Query Items` endpoint using `encrypted_query` parameter to get items matching the encrypted query.

Parameters:
  • tenant – User tenant.
  • project_id – Project identifier.
JSON Data:

Data is passed in as a JSON object.

Valid object keys:

    • query – Search query to be encrypted.
    • aggregations – Dictionary of aggregations to execute on matching items to be encrypted.
    • created_before – Date and Time value limiting the returned items to items that have been created before this date to be encrypted.
    • created_after – Date and Time limiting the returned items to items that have been created after this date to be encrypted
    • options – Dictionary of options influencing the result-set to be encrypted.
      • Currently, only the 'template_params' property of 'options' dict is encrypted even though the 'options' parameter support much more variables with the `Query Items` resource
    • fields – Comma-separated list of fields to be encrypted.

Example request body:

{
	"query": "Hitachi"
} 
Headers:See Common Headers.
Status Codes:
  • 200 – Results returned.

See also Common Status Codes.

Returns:

Response fields:

  • encrypted_query – Encrypted version of `query` and any other parameters ('aggregations', 'options') if specified.

{
	'encrypted_query': 'YR4h147YAldsARmTmIrOcJqpuntiJULXPV3ZrXblVWvbCavvESTw4Jis6sTgGC9a1LhrLd9Nq-77CNX2eeieMEDnPFPRqlPGO8V'
}

List Items (deprecated)

GET https://squirro-endpoint/api/topic/v0/(string: tenant)/projects/(id: project_id)/items

Returns a selection of items from the project. The items be filtered with a query and other parameters.

Parameters:
  • tenant – User tenant.
  • project_id – Project identifier.
Query Parameters:
  • start – the index of the first item to be returned, useful for pagination (default: 0). Since the returned items are filtered after pagination, the start returned in the next_params field should be passed in to avoid receiving duplicate items.
  • count – the number of items returned (default 15). Since the returned items are filtered after pagination, the number of returned items may be lower than this number.
  • created_before – Date and Time value which limits the returned items to items that have been created before this date.
  • created_after – Date and Time value which limits the returned items to items that have been created after this date.
  • object_ids – Comma-separated list of object IDs from which to return items.
  • query – Search for items that match this query. For each item a score is calculated which reflects how well it matches this query.
  • fingerprint_type_id – If set, the fingerprint with this id and of type fingerprint_type and with name fingerprint_name is applied to this request. Set all three fingerprint parameters for this to work.
  • fingerprint_type – The type of the requested fingerprint to apply. Set this parameter together with fingerprint_type_id and fingerprint_name.
  • fingerprint_name – The name of the requested fingerprint to apply. Set this parameter together with fingerprint_type_id and fingerprint_name.
  • filter_query – Filters the resulting items additionally with a query. Each item returned must match this query. The syntax follows the Query Syntax.
  • enable_query_highlighting – Boolean value. Allows to highlight terms of a user query found in the items of the result set. The found search terms will be wrapped with a pre_tag and a post_tag.
  • highlight_smartfilters – Allows to highlight terms of an applied Smart Filter found in the items of the result set. The found search terms will be wrapped with a pre_tag and a post_tag.
  • noise_level – If set only return the most relevant items for the provided noise level which is a float between 0.0 (only the most relevant items) and 1.0 (all items). A hint of how many items are to be retrieved for noise level 1.0 can be passed in with expected_num_results.
  • expected_num_results – Number of results to be expected for the provided query. Can be used as a hint in conjunction with noise_level. This should be the expected_num_results returned in the next_params field of the response.
  • languages – Restrict the returned items to one or many languages. Can be a comma separated list of language codes or the special string user to this users configured languages. If not set, items in all languages are returned.
  • location – Restrict the returned items to a specific geographic region. This parameter can take three different forms: ‘lat,lon’ for searching on one spot, ‘lat,lon,radius’ to search within a circle (radius unit is kilometer) or ‘lat1,lon1,lat2,lon2’ to search in an envelope (top right, bottom left)
  • fields – Comma-separated list of fields to return. If this is specified, the return value is limited to only these fields. See the Item Format reference for a list of available fields.
  • keyword_facets – Boolean value. If set to true, keyword facets are calculated. For each keyword the top values in the result set are calculated and returned with the query result.
  • facet_fields – Comma-separated list of fields for which to calculate the facets. Defaults: all fields.
  • nof_facets – Numeric value indicating how many of the top facet values should be returned. Default: 10.
  • date_histogram – Boolean value. If set to true, the date distribution of the query result is calculated and returned.
  • date_histogram_fields – Comma-separated list of fields for which values you would like to receive date histogram information.
  • filter_related_items – Boolean value. Set to true to remove near-duplicates from the result list.
Headers:See Common Headers.
Status Codes:
  • 200 – Results returned.

See also Common Status Codes.

Returns:

Results including some pagination information.

Response fields:

  • items – List of items.
  • count– The number of items returned with this result set.
  • total – How many items match the current query.
  • eof – True if the there are no further results available to paginate to.
  • next_params – Query parameters that should be passed in with the next request when paginating.


{
    "count": 1,
    "now": "2014-12-17T14:27:47",
    "next_params": {
        "start": 1,
        "expected_num_results": 92
    },
    "eof": false,
    "items": [
        {
            "body": "<html><body>
<p class=\"tweet\">ABB and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system…",
            "language": "en",
            "title": "Supreme & Co.",
            "read": false,
            "created_at": "2014-12-17T13:10:45",
            "modified_at": "2014-12-17T13:11:16",
            "related_items": [],
            "id": "Fd2yF5BvQX2LlCdYPZ-Zow",
            "sources": [
                {
                    "link": "https://twitter.com/search?q=abb -is:retweet&f=realtime",
                    "title": "Twitter search for: \"abb\"",
                    "object_ids": [
                        "w--MRsdxR26LlymevPTpMA"
                    ],
                    "id": "nN3pr5BwQJevxbSzMB9Www",
                    "provider": "twitter"
                }
            ],
            "score": null,
            "link": "https://www.facebook.com/pages/Supreme-Co/356890661079035?ref=hl",
            "objects": [
                {
                    "source_ids": [
                        "nN3pr5BwQJevxbSzMB9Www"
                    ],
                    "project_id": "D9omq-WVQ0O2hxYp9fKUlA",
                    "type": "default",
                    "id": "w--MRsdxR26LlymevPTpMA",
                    "title": "default"
                }
            ],
            "keywords": {
                "from": [
                    "supremeco"
                ],
                "Sentiment": [
                    "Neutral"
                ],
                "Geo_State": [
                    "Japan"
                ],
                "verb": [
                    "post"
                ],
                "Organization": [
                    "ABB Group",
                    "Hitachi"
                ]
            },
            "starred": false,
            "external_id": "545204453427531777",
            "abstract": [
                "ABB and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system solutions in Japan.\ntinyurl.com/nzhqzrfABB",
                " and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system solutions in Japan.\nBusiness Standard reported that ABB",
                " and <squirro:highlight>Hitachi</squirro:highlight> entered into a JV for providing high voltage direct current system solutions in Japan",
                "-sales services related to the DC system of HVDC projects. <squirro:highlight>Hitachi</squirro:highlight> and ABB will take equity..."
            ]
        }
    ],
    "facets": {},
    "field_histograms": {},
    "total": 92,
    "fingerprint_matches": {}
} 

Get Item

GET https://squirro-endpoint/api/topic/v0/(string: tenant)/projects/(id: project_id)/items/(id: item_id)

Returns a single item from the project.

Parameters:
  • tenant – User tenant.
  • project_id – Project identifier.
  • item_id – Item identifier.
Query Parameters:
  • highlight_query – Query for which the item is to be highlighted.
  • explain_smartfilters – Comma-separated list of Smart Filters for which to include explanations in query result.
  • fields – Comma-separated list of fields to return. If this is specified, the return value is limited to only these fields. See the Item Format reference for a list of available fields.
Headers:See Common Headers.
Status Codes:
  • 200 – Item returned.

See also Common Status Codes.

Returns:

The item in the item response field.

{
    "item": {
        "body": "<html><body><p class=\"tweet\">ABB and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system…",
        "language": "en",
        "title": "Supreme & Co.",
        "read": false,
        "created_at": "2014-12-17T13:10:45",
        "modified_at": "2014-12-17T13:11:16",
        "related_items": [],
        "id": "Fd2yF5BvQX2LlCdYPZ-Zow",
        "sources": [
            {
                "link": "https://twitter.com/search?q=abb -is:retweet&f=realtime",
                "title": "Twitter search for: \"abb\"",
                "object_ids": ["w--MRsdxR26LlymevPTpMA"],
                "id": "nN3pr5BwQJevxbSzMB9Www",
                "provider": "twitter"
            }
        ],
        "score": null,
        "link": "https://www.facebook.com/pages/Supreme-Co/356890661079035?ref=hl",
        "objects": [
            {
                "source_ids": ["nN3pr5BwQJevxbSzMB9Www"],
                "project_id": "D9omq-WVQ0O2hxYp9fKUlA",
                "type": "default",
                "id": "w--MRsdxR26LlymevPTpMA",
                "title": "default"
            }
        ],
        "keywords": {
            "from": ["supremeco"],
            "Sentiment": ["Neutral"],
            "Geo_State": ["Japan"],
            "verb": ["post"],
            "Organization": ["ABB Group", "Hitachi"]
        },
        "starred": false,
        "external_id": "545204453427531777",
        "abstract": [
            "ABB and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system solutions in Japan.\ntinyurl.com/nzhqzrfABB",
            " and <squirro:highlight>Hitachi</squirro:highlight> forms a JV to provide HVDC system solutions in Japan.\nBusiness Standard reported that ABB",
            " and <squirro:highlight>Hitachi</squirro:highlight> entered into a JV for providing high voltage direct current system solutions in Japan",
            "-sales services related to the DC system of HVDC projects. <squirro:highlight>Hitachi</squirro:highlight> and ABB will take equity..."
        ]
    }
}

Update Item

PUT https://squirro-endpoint/api/topic/v0/(string: tenant)/projects/(id: project_id)/items/(id: item_id)

Updates an item in the project. This can only be used to modify the keywords and states.

Parameters:
  • tenant – User tenant.
  • project_id – Project identifier.
  • item_id – Item identifier.
JSON Data:

Data is passed in as a JSON object.

Valid object keys:

  • state – Dictionary with the new item states. Valid states are: read, starred. States that are not specified are not modified.
  • keywords – Dictionary with keywords to update. This can only be used on projects which have their own project index. Keywords that are not specified are removed from the item.

Example request body:

{
    "state": {
        "read": true,
        "starred": false
    },
    "keywords": {
        "Author": ["John Doe"]
    }
}
Headers:See Common Headers.
Status Codes:
  • 204 – Item modified.

See also Common Status Codes.

Delete Item

DELETE https://squirro-endpoint/api/topic/v0/(string: tenant)/projects/(id: project_id)/items/(id: item_id)

This resource can only be used on projects which have their own project index

This resource

Parameters:
  • tenant – User tenant.
  • project_id – Project identifier.
  • item_id – Item identifier.
Headers:See Common Headers.
Status Codes:
  • 204 – Item deleted.

See also Common Status Codes.

Recommend similar items

GET/POST https://squirro-endpoint/api/topic/v0/(string: tenant)/projects/(id: project_id)/items/recommend

Returns a recommendation of similar items from the project given an item id or text.

Parameters:
  • tenant – User tenant.
  • project_id – Project identifier.
JSON Data:

Data is passed in as a JSON object.

Valid object keys:

  • item_id – ID of item used for recommendation (optional).
  • external_id – External ID of item used for recommendation if item_id is not provided (optional)
  • text – Text content used for recommendation if neither item_id nor external_id are not provided (optional)
  • method – Recommendation method (optional).
    • These values are supported:
      • smartfilter: Use squirro Smart Filter method (default)
      • mlt: Use "More like this" of elasticsearch
      • tfidf: Use first 100 words of document to find similar documents based on tfidf score.
  • related_fields – Fields used to find relationship for between items for recommendation. If this parameter is not set we use the text in title, body and abstract  of item
  • count – Maximum number of items to return.
  • fields – Fields to return.
  • options –

    Dictionary of options that influence the result-set. Valid options are:

    • fold_near_duplicates to fold near-duplicates together and filter them out of the result-stream. Defaults to False.
  • created_before – Restrict result set to items created before created_before.
  • created_after – Restrict result set to items created after created_after.
  • query – Search query to restrict the recommendation set.
  • aggregations – Aggregation of faceted fields
Headers:See Common Headers.
Status Codes:
  • 200 – Results returned.

See also Common Status Codes.

Returns:

Results.

Response fields:

  • items – List of items.
  • now – Timestamp for the event of response generation.
  • count – The number of items returned with this result set.
  • total – Total number of items in recommendation result
  • method – Method used for recommendation
  • master_item – Original item used to get recommendation
  • time_ms – Time of execution main component of recommendation in millisecond.
{
  "count": 2,
  "total": 5,
  "items": [
    {
      "title": "How Do I Get a Mortgage?",
      "read": false,
      "created_at": "2017-04-26T19:02:17",
      "related_items": [
         
      ],
      "score": 37.719635,
      "link": "https://www.nytimes.com/interactive/2017/realestate/How-Do-I-Get-a-Mortgage.html",
      "external_id": "5900eebc95d0e024b586d7d8",
      "id": "CPIalbwkobYkwjwzzkWcGw"
    },
    {
      "title": "How Do I Get a Good Interest Rate?",
      "read": false,
      "created_at": "2017-04-26T19:04:01",
      "related_items": [
         
      ],
      "score": 26.451519,
      "link": "https://www.nytimes.com/interactive/2017/realestate/How-Do-I-Get-a-Good-Interest-Rate.html",
      "external_id": "5900ef2a95d0e024b586d7dc",
      "id": "YiBhdq04vzu9nSkDK05GAw"
    },
 
  ],
  "master_item": {
    "title": "What Kind of Mortgage Should I Get?",
    "read": true,
    "created_at": "2017-04-26T19:03:03",
    "related_items": [
       
    ],
    "score": null,
    "link": "https://www.nytimes.com/interactive/2017/realestate/What-Kind-of-Mortgage-Should-I-Get.html",
    "external_id": "5900eeea95d0e024b586d7d9",
    "id": "nul6nurQKGOJIdphYRYAjg"
  },
  "time_ms": {
    "query": 31,
    "master_item": 268
  },
  "now": "2017-11-07T09:04:49.699453",
  "method": "mlt"
}

Recommendation methods

GET https://squirro-endpoint/api/topic/v0/(string: tenant)/projects/(id: project_id)/items/recommend/methods

Returns the supported recommendation methods for items from the project.

Parameters:
  • tenant – User tenant.
  • project_id – Project identifier.
Headers:See Common Headers.
Status Codes:
  • 200 – Methods returned.

See also Common Status Codes.

Returns:

Method names.

Response fields:

  • methods – List of method names.
  • default – The default that would be chosen for this project.
{
    "methods": ["smartfilter", "mlt", "tfidf"],
    "default": "smartfilter"
}