Table of Contents
| Table of Contents |
|---|
Overview
This page outlines a number of steps that can be taken in order to configure the Squirro environment in a more secure manner than out of the box. This includes Squirro-specific steps, such as encrypting configuration files or enabling single sign-on (SSO), but also shows how the operating systems and some of the shipped services can be configured in a more restrictive way.
Pre-Requisites
Linux instance to be setup according to third party standards and protocols.
Familiarity with Squirro services and setup procedures.
Linux & Networking
Firewall Settings
Squirro Ports
Incoming traffic to Squirro servers are open on a number of TCP ports. The individual ports and the required access level are documented below.
Storage Nodes
On Squirro storage nodes all port except the following can be blocked.
...
TCP Port
...
Usage
...
Open for
...
9200
...
Elasticsearch access
...
All storage and Squirro nodes
...
9300
...
Elasticsearch replication
...
All storage nodes
Squirro Nodes
On Squirro cluster nodes all ports except the following can be blocked.
...
TCP Port
...
Usage
...
Open for
...
443
...
Web and API access (SSL-protected)
...
All clients (or load balancer)
...
2181
...
Zookeeper
...
All Squirro nodes
...
2888
...
Zookeeper node discovery
...
All Squirro nodes (in multi-node setups)
...
3306
...
MariaDB
...
All Squirro nodes
...
3888
...
Zookeeper node discovery
...
All Squirro nodes (in multi-node setups)
...
6379
...
Redis (storage)
...
All Squirro nodes
...
6380
...
Redis (cache)
...
All Squirro nodes
For more information on configuring ports, please visit:
Nginx
Squirro ships with an Nginx web server which terminates web connections and then forwards them internally on the machine to the services.
SSL certificate
For the HTTPS connection to the Squirro cluster node, a self-signed SSL certificate is generated by default. That certificate will trigger warnings for users and is not trusted by the Squirro Toolbox either. For security reasons, an SSL certificate should be issued by the respective body. For more information on configuring SSL certificates please visit:
Limit TLS protocols
As a compromise to security and compatibility, Squirro’s default configuration of Nginx allows TLS 1.0, 1.1 and 1.2. This can be reduced to only allow TLS 1.2 if the environment (users and clients connecting) allows that restriction.
To change, edit /etc/nginx/conf.d/ssl.inc. Look for the ssl_protocols and change it to:
| Code Block |
|---|
ssl_protocols TLSv1.2; |
The Red Hat Enterprise Linux 7 version of nginx does not yet support TLS 1.3.
Remove non-secure ports
The default configuration listens on port 80 and 81 for HTTP and 443 for HTTPS. The ports 80 and 81 can be removed to avoid these connections (requiring further changes in the Squirro configuration below).
Edit /etc/nginx/conf.d/00-default.conf and remove the following two lines:
| Code Block |
|---|
listen 80;
listen 81; |
Enable secure flag of cookies
The secure flag in cookies can get enabled to make HTTPS connections more secure. This prevents for cookie to get transmitted in clear-text if the user visits any HTTP URLs within the scope of the cookie.
Edit /etc/squirro/frontend.ini and add the following line in the section [flask_app]:
| Code Block |
|---|
session_cookie_secure = true |
MariaDB
SSL certificate
MariaDB can be configured to encrypt connections with TLS. For this, an SSL certificate needs to be provided at /etc/my.cnf.d/certificates/. To create self-signed certificates, the official documentation for Certificate Creation with OpenSSL can be followed.
The files can then be provided to MariaDB by putting the following configuration into /etc/my.cnf.d/security.cnf:
| Code Block |
|---|
[server]
ssl_cert = /etc/my.cnf.d/certificates/server-cert.pem
ssl_key = /etc/my.cnf.d/certificates/server-key.pem
ssl_ca = /etc/my.cnf.d/certificates/ca.pem |
Enabling SSL
You must edit /etc/squirro/common.ini and add the following line ssl=true in the db section as follow:
| Code Block |
|---|
[db]
mysql_transaction_isolation=READ COMMITTED
ssl=true |
Changing user passwords [Squirro Version >= 3.1.0]
Each Squirro service with a database has a dedicated MariaDB user. Squirro recommends the default passwords of these users are changed. This can be achieved using the following commands per service:
| Code Block |
|---|
mysql -u root -p
USE mysql;
UPDATE user SET password=PASSWORD('YourPasswordHere') WHERE User='<service_user_name>';
FLUSH PRIVILEGES; |
The service database users for which to do this are:
configuration
cluster (will only exist on multi-node installations)
datasource
emailsender
filtering
fingerprint
machinelearning
plumber
scheduler
topic
trends
user
The password can be different for each user.
Redis
There are two Redis instances, one for queueing and more persistent metadata, and one for caching. Their respective configuration files are:
Storage:
/etc/redis/redis.confCache:
/etc/redis/cache.conf
SSL certificate
The Redis version shipped with Squirro does not currently allow SSL encryption. Later in 2020 Squirro aims to include Redis 6, which provides TLS encryption.
Changing Redis Passwords
Squirro recommends the default passwords of the Redis instances be changed. In both configuration files, the password can be set on the lines:
| Code Block |
|---|
masterauth <password>
…
requirepass <password> |
The password for the two Redis instances should be the same.
Elasticsearch [Squirro Version >= 3.1.0]
SSL certificate
Elasticsearch allows encryption of communication with TLS.
For this the certificate files can be put into /etc/elasticsearch/certs as:
servername.keyservername.crtca.crt
They are then referenced in /etc/elasticsearch/elasticsearch.yml by adding the following lines:
| Code Block |
|---|
xpack.ssl.key: certs/servername.key
xpack.ssl.certificate: certs/servername.crt
xpack.ssl.certificate_authorities: certs/ca.crt
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true |
Full documentation is available in the Elasticsearch manual at Encrypting Communications.
Access to Elasticsearch is passed through Nginx as well. To use HTTPS, the configuration needs to be adjusted in /etc/nginx/conf.d/00-default.conf to access https://elastic instead of http://elastic. This can be achieved with the following command on the shell:
| Code Block |
|---|
sed -i'' -e 's@http://elastic@https://elastic@' /etc/nginx/conf.d/00-default.conf |
To change the connection pattern in Elasticsearch you must edit /etc/squirro/common.ini and change the following line:
| Code Block |
|---|
es_index_servers = http://127.0.0.1:81/ext/elastic
To:
es_index_servers = https://127.0.0.1:81/ext/elastic |
Password
A password can be defined to access Elasticsearch. For this the following line needs to be added to /etc/elasticsearch/elasticsearch.yml:
| Code Block |
|---|
xpack.security.enabled: true |
The password can then be defined by calling the command:
| Code Block |
|---|
/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive |
After you set your password, you are required to add this parameter to squirro, you must edit /etc/squirro/common.ini and add the following lines (under [index]):
| Code Block |
|---|
es_auth_user = elastic # or the user you selected
es_auth_password = YOUR_PASSWORD |
NB: you can also generate a random password by calling:
| Code Block |
|---|
/usr/share/elasticsearch/bin/bin/elasticsearch-setup-passwords auto -b |
important: on ES version < 7.7.1 you must first install x-pack and then the commands change as follow:
| Code Block |
|---|
/usr/share/elasticsearch/bin/bin/x-pack/elasticsearch-setup-passwords auto -b
/usr/share/elasticsearch/bin/x-pack/setup-passwords interactive |
Cluster references:
https://www.elastic.co/guide/en/elasticsearch/reference/7.8/configuring-tls.html#tls-transport
Squirro
SSO
Squirro supports the use of Single Sign-On with multiple SAML protocols. Squirro recommends that is enabled for secure server setups. SSO can be enabled from the /etc/squirro/frontend.ini
File: /etc/squirro/frontend.conf
| Code Block |
|---|
[security]
sso_enabled = true
sso_endpoint = http://localhost:81/studio/extauth_saml/extauth |
For more information on configuring SSO please visit:
SAML Single Sign-On for Squirro
Making session non-permanent
By default, Squirro sessions are permanent. Especially in the context of SSO, this should be limited to browser session timeouts. In order to make them non-permanent, settings within the /etc/squirro/frontend.ini can be modified accordingly:
| Code Block |
|---|
[frontend]
session_permanent = false |
For more information on configuring SSO sessions, please visit:
SAML Single Sign-On for Squirro
Changing database passwords
The changed Redis and MariaDB passwords need to be updated in the configuration files. Where a Vault is used (see “Encrypting Configuration Files” section), the passwords can instead be set in the corresponding values there.
...
Filename
...
INI file section
...
INI file key
...
Value
...
Redis
...
/etc/squirro/cluster.ini
...
redis
...
password
...
Redis password
...
/etc/squirro/common.ini
...
redis
...
password
...
Redis password
...
/etc/squirro/common.ini
...
redis_content
...
password
...
Redis password
...
/etc/squirro/common.ini
...
redis_cache
...
password
...
Redis password
...
/etc/squirro/common.ini
...
redis_http_cache
...
password
...
Redis password
...
/etc/squirro/common.ini
...
redis_provider_deduplication
...
password
...
Redis password
...
/etc/squirro/common.ini
...
redis_bloom
...
password
...
Redis password
...
/etc/squirro/common.ini
...
redis_studio
...
password
...
Redis password
...
/etc/squirro/common.ini
...
redis_studio_cache
...
password
...
Redis password
...
/etc/squirro/datasource.ini
...
queues_local
...
redis_password
...
Redis password
...
/etc/squirro/datasource.ini
...
redis_key_value_cache_provider
...
redis_password
...
Redis password
...
/etc/squirro/datasource.ini
...
redis_key_value_store_provider
...
redis_password
...
Redis password
...
/etc/squirro/filtering.ini
...
redis_email_history
...
password
...
Redis password
...
/etc/squirro/filtering.ini
...
redis_item_batching
...
password
...
Redis password
...
/etc/squirro/frontend.ini
...
redis_studio
...
password
...
Redis password
...
/etc/squirro/frontend.ini
...
redis_studio_cache
...
password
...
Redis password
...
/etc/squirro/machinelearning.ini
...
queues_local
...
redis_password
...
Redis password
...
/etc/squirro/relatedstory.ini
...
redis_duplicate
...
password
...
Redis password
...
/etc/squirro/relatedstory.ini
...
redis_nearduplicate
...
password
...
Redis password
...
MariaDB
...
/etc/squirro/configuration.ini
...
configuration
...
db
...
MariaDB password for user configuration
...
/etc/squirro/cluster.ini
...
configuration
...
db
...
MariaDB password for user cluster
...
/etc/squirro/datasource.ini
...
datasource
...
db
...
MariaDB password for user datasource
...
/etc/squirro/emailsender.ini
...
emailsender
...
db
...
MariaDB password for user emailsender
...
/etc/squirro/filtering.ini
...
filtering
...
db
...
MariaDB password for user filtering
...
/etc/squirro/fingerprint.ini
...
filtering
...
db
...
MariaDB password for user fingerprint
...
/etc/squirro/machinelearning.ini
...
machinelearning
...
db
...
MariaDB password for user machinelearning
...
/etc/squirro/plumber.ini
...
plumber
...
db
...
MariaDB password for user plumber
...
/etc/squirro/scheduler.ini
...
scheduler
...
db
...
MariaDB password for user scheduler
...
/etc/squirro/topic.ini
...
topic
...
db
...
MariaDB password for user topic
...
/etc/squirro/topicproxy.ini
...
topicproxy
...
db
...
MariaDB password for user topic
...
/etc/squirro/trends.ini
...
trends
...
db
...
MariaDB password for user trends
...
/etc/squirro/user.ini
...
user
...
db
...
MariaDB password for user user
...
/etc/squirro/userproxy.ini
...
userproxy
...
db
...
MariaDB password for user user
Changing Endpoints for APIs
If the HTTP ports have been disabled, Squirro needs to be changed to use the HTTPS endpoints. For this, edit /etc/squirro/common.ini and change any occurrence of http://127.0.0.1:81/ to https://127.0.0.1/.
This can be achieved with the following command:
| Code Block |
|---|
sed -i'' -e 's@http://127.0.0.1:81@https://127.0.0.1@' /etc/squirro/common.ini |
Changing MariaDB connection to TLS
SSL can be enabled for MariaDB connections by setting the following configuration value in /etc/squirro/common.ini:
| Code Block |
|---|
[db]
ssl=true |
The [db] section will already exist in the config file, thus the full section will then look something like this:
| Code Block |
|---|
[db]
mysql_transaction_isolation=READ COMMITTED
ssl=true |
Changing Default Session Passwords
Squirro recommends that the below session passwords stored in /etc/squirro/frontend.ini be changed from the default values:
| Code Block |
|---|
[frontend]
# key used to create secure HMAC tokens from passwords
password_key = …
[flask_app]
secret_key = … |
Random values can be generated for these.
Encrypting Configuration Files
By default passwords (e.g. the passwords for Redis or MariaDB) are configured in plain text in the configuration files. However, Squirro can provide a mechanism so that this sensitive information is encrypted. There are three different ways to encrypt Squirro configuration files:
Encryption key in a Unix environment variable
Encryption key stored in a file
A custom command for encrypting and decrypting
For more information on the different types of encryption, please visit:
Securing your Configuration Files
Should there be some form of the vault where credentials are stored, Squirro can provide the necessary decryption tools in order to decrypt values.
Content Security Policy (CSP)
The Content Security Policy for Squirro content can be defined in the server configurations. If for instance image resources that are references from within Squirro items are not loading, you can adapt the CSP accordingly to enable those cross-site references. For more information see below links.
Relevant Links:
...
https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
...
now be found at Secure Configuration Guide on the Squirro Docs site.