Page Comparison

Table of Contents

Introduction

Squirro uses RPM packages to install on Linux systems. One functionality of these packages is, that they install configuration files. These files are usually located in the /etc directory and can be edited to change the configuration of the installed services.

In Squirro's case, the following directories contain configuration files relevant to the running of its services:

• /etc/squirro - configuration for the Squirro microservices
  
• /etc/nginx/conf.d - web server configuration to serve the Squirro application and microservices
• /etc/elasticsearch - Elasticsearch configuration
• /etc/sysconfig - Squirro environment settings (filename: squirro)

Identifying Conflicts

If you have edited any of the files in these directories and upgrade Squirro, this may result in .rpmnew files being written.

You can find those files with the following command:

find /etc -name '*.rpmnew'

This command will find all the configuration conflicts, not just the ones for Squirro services. Use the folder list above as reference if you only want to identify the Squirro configuration conflicts.

Resolve File

For each file that has been identified in the previous section, follow this process to resolve any conflicts.

In the example we assume that the find command returned the file /etc/squirro/test.ini.rpmnew.

1. Change into the right directory: cd /etc/squirro
2. Compare the rpmnew file to the original: diff -u test.ini test.ini.rpmnew
3. Now there are two ways to resolve this conflict:
   • If you want to discard all your changes, you can simply move the rpmnew file over the old file: mv test.ini.rpmnew test.ini
   • To manually copy some of the configuration changes from the rpmnew file to your configuration file, edit the file: nano test.ini or vi test.ini

Usage of Diff

To compare the files, the Unix utility diff can be used. The recommended usage is a follows:

diff -u test.ini test.ini.rpmnew

This outputs the changed lines prefixed with + for lines that have been added in the rpmnew file compared to the old configuration and prefixed with - for removed lines. Additionally some lines before and after each change are included for better context.

When resolving these conflicts, proceed as follows:

• Ignore any of your changes - even though they are flagged as one addition and one removal. For example if you changed the database password, diff will flag an addition of the default database string and removal of your custom database string.
• Add any new configuration options that are not present in your config file.
• Uncomment or remove any removed configuration options. You can comment out lines by prefixing them with the # character.

Multi-node Squirro Clusters

This section highlights configuration settings specific to multi-node Squirro clusters. If you run Squirro on a single system, you can safely skip this section. On multi-node clusters, Squirro services do not always use the local MySql or Redis servers, but to one of the servers that acts as the "master". As a result all settings called either db_endpoint_discovery (for MySql connections) or endpoint_discovery (for Redis) need to be set to true. In contrast single-node Squirro installations as well as brand-new installations (or upgrades introducing new Squirro services or to make this even more confusing existing Squirro services whose new version uses a new MySql or Redis database) introduce out-of-the-box settings of false for both endpoint_discovery and db_endpoint_discovery.

Here is an example based on the Squirro Topic-Api service corresponding to the Squirro 2.4.2 release:

...

[handler_file]
application = pipeline

[pipeline]
item_debug_output_directory = /var/lib/squirro/pipeline/debug

[fetch]
fetch_directory = /var/cache/squirro/pipeline/pages

[redis_bloom]
endpoint_discovery = false
password = ...

[redis_in_flight]
endpoint_discovery = false
password = ...

[redis_pipeline_retries]
endpoint_discovery = false
password = ...

[queues_local]
endpoint_discovery = false
redis_password = ...

[handler_file]
application = pipeline

[pipeline]
item_debug_output_directory = /var/lib/squirro/pipeline/debug

[fetch]
fetch_directory = /var/cache/squirro/pipeline/pages

[redis_bloom]
endpoint_discovery = true
password = ...

[redis_in_flight]
endpoint_discovery = true
password = ...

[redis_pipeline_retries]
endpoint_discovery = true
password = ...

[queues_local]
endpoint_discovery = true
redis_password = ...

Guidance for resolving differences in multi-node clusters are:

...

This page can now be found at Upgrading Configuration Conflicts on the Squirro Docs site.