In some cases, e.g. when deploying Squirro to Amazon AWS or MS Azure, you may elect to rely on a remote MySql Server installation (e.g. RDS in case of AWS) and/or remote Redis Servers (e.g. RedisLabs). This page describes how you would configure such remote servers for use by Squirro and how to configure Squirro Cluster Nodes accordingly.
Table of Contents
| Table of Contents | ||||
|---|---|---|---|---|
|
Preparation
Before you install Squirro RPMs, ensure that you put in place the file /etc/squirro/backends.ini with the following content:
To set up Squirro with remote MySql server and Redis server "Backends", please create a readable file /etc/squirro/backends.ini :
| Code Block |
|---|
mkdir -p /etc/squirro
touch /etc/squirro/backends.ini |
And add the following content depending on whether you want to run MySql server or Redis servers remotely or both. The example below show the content for both MySql and Redis servers to be configured remotely:
| Code Block | ||
|---|---|---|
| ||
# Rely on a remote MySql Server installation
is_mysql_server_remote = true
# Rely on remote Redis Servers
is_redis_server_remote = true |
MySql Server Setup
On the MySql Server front there are server-level configurations, database-level configurations, and Squirro Cluster Node-level configurationsServer-level configuration
MySql Server-level Configuration
Configure MySql on User level scope, make sure your ~/.my.cnf file content looks like this:
| Code Block |
|---|
[client]
host="remote mysql server"
user="user for the remote mysql server"
password="password for the remote mysql server"
ssl_ca="location of your SSL certificate" |
If there is a requirement, follow the instruction here to secure MySql connection with SSL.
Following configuration are required by Squirro on MySql server level.
| Code Block |
|---|
[mysqld]
character_set_server=utf8
collation_server=utf8_unicode_ci
innodb_flush_log_at_trx_commit=1 |
Make sure you configure the MySql client on all Squirro Cluster Nodes to point to the server installation. Also note the name of the MySql administrator account - often called root.
MySql Database and User Creation
Run the following on a Squirro Cluster Node on which /etc/squirro/backends.ini has been set up and squirro-cluster-node has been installed per Setup on Linux:
| Code Block |
|---|
MYSQL_ADMIN=root # or the name of the MySql Administrator account able to create users and databases
DATABASE_PASSWORD="password of your choosing"
squirro_mysql_databases=`cd /etc/squirro/; grep "^db *=" *.ini | sed -e "s/\..*$//"`
for mysql_db in $squirro_mysql_databases; do
. /opt/squirro/setup/ensure_mysql_user_and_database_exist.sh $MYSQL_ADMIN $mysql_db $DATABASE_PASSWORD
done |
Squirro Cluster Node Configuration
Before starting Squirro services, the configuration files under /etc/squirro/ need to be made to point to the remote installation:
| Code Block |
|---|
MYSQL_SERVER_ADDRESS="address or name of MySql installation"
sed -e "s|squirro/4u|$DATABASE_PASSWORD|" -e "s|localhost|$MYSQL_SERVER_ADDRESS|" -i /etc/squirro/*.ini |
Note that the script above depends on the same DATABASE_PASSWORD variable from the "MySql Database and User Creation" section above.
Redis Server Setup
Redis Server-level Configuration
Squirro writes into about a dozen Redis databases. Ideally your Redis Server setup would have multiple Redis databases although somewhat experimentally it should be possible to use a single Redis Server. We do however recommend at least having two Redis-Servers one for key/value storage and other for caching. If required, follow the instruction to Secure Redis instance over SSL.
Squirro Cluster Node Configuration
You would point the Squirro services to the two Redis Server installation like so:
| Code Block |
|---|
REDIS_STORAGE_HOST="Redis Server Storage host"
REDIS_STORAGE_PORT="Redis Server Storage port number"
REDIS_STORAGE_PASSWORD="Redis Server Storage password"
REDIS_CACHE_HOST="Redis Server Cache host"
REDIS_CACHE_PORT="Redis Server Cache port number"
REDIS_CACHE_PASSWORD="Redis Server Cache password"
redis_storage_dbs=`grep "^\[redis.*]" /etc/squirro/*.ini | grep -v "_cache.*]" | grep -v "redis_key_value" | sed -e "s/^.*\[//" -e "s/] *$//" | sort -u`
redis_cache_dbs=`grep "^\[redis.*_cache.*]" /etc/squirro/*.ini | grep -v "redis_key_value" | sed -e "s/^.*\[//" -e "s/] *$//" | sort -u`
redis_key_value_dbs=`grep "^\[redis_key_value_store.*]" /etc/squirro/*.ini | sed -e "s/^.*\[//" -e "s/] *$//" | sort -u`
redis_key_value_cache_dbs=`grep "^\[redis_key_value_cache.*]" /etc/squirro/*.ini | sed -e "s/^.*\[//" -e "s/] *$//" | sort -u`
# remove old redis passwords:
sed -e "/^password *=/d" -i /etc/squirro/*.ini
sed -e "/^redis_password *=/d" -i /etc/squirro/*.ini
for storage_db in $redis_storage_dbs; do
sed "s|^\(\[${storage_db}] *\)$|\1\nhost = $REDIS_STORAGE_HOST\nport = $REDIS_STORAGE_PORT\npassword = $REDIS_STORAGE_PASSWORD|" -i /etc/squirro/*.ini
done
for cache_db in $redis_cache_dbs; do
sed "s|^\(\[${cache_db}] *\)$|\1\nhost = $REDIS_CACHE_HOST\nport = $REDIS_CACHE_PORT\npassword = $REDIS_CACHE_PASSWORD|" -i /etc/squirro/*.ini
done
for storage_db in $redis_key_value_dbs; do
sed "s|^\(\[${storage_db}] *\)$|\1\nredis_host = $REDIS_STORAGE_HOST\nredis_port = $REDIS_STORAGE_PORT\nredis_password = $REDIS_STORAGE_PASSWORD|" -i /etc/squirro/*.ini
done
for cache_db in $redis_key_value_cache_dbs; do
sed "s|^\(\[${cache_db}] *\)$|\1\nredis_host = $REDIS_CACHE_HOST\nredis_port = $REDIS_CACHE_PORT\nredis_password = $REDIS_CACHE_PASSWORD|" -i /etc/squirro/*.ini
done
if grep -Fxq "[queues_local]" /etc/squirro/common.ini
then
sed "s|^\(\[queues_local] *\)$|\1\nredis_host = $REDIS_STORAGE_HOST\nredis_port = $REDIS_STORAGE_PORT\nredis_password = $REDIS_STORAGE_PASSWORD|" -i /etc/squirro/common.ini
else
cat >>/etc/squirro/common.ini <<FOO
[queues_local]
redis_host = $REDIS_STORAGE_HOST
redis_port = $REDIS_STORAGE_PORT
redis_password = $REDIS_STORAGE_PASSWORD
FOO
fi |
Return to the Linux Installation Steps
...
This page can now be found at Installing Squirro on Linux on the Squirro Docs site.