Fresh Installation Instructions

Please follow the regular installation steps

Upgrade Instructions

From earlier versions 2.5.2-4031 or lower

If you have already upgraded to 2.5.2 before 9th November 2017 and you are intending to apply a patch release of storage node, after normal steps of upgrading storage node, you please run this command:

bash /opt/squirro/elasticsearch/update/migrate-storage-node.sh

From version 2.5.1

1. Upgrade Storage Nodes and Cluster Nodes collocated on the same machine/VM

for service in `ls /etc/monit.d/sq*d | sed -e "s|^.*/||" | grep -v "sqclusterd"`; do monit stop $service; done
# wait for `monit summary` to indicate that all but 5 services are stopped
yum update squirro-storage-node-users
yum update elasticsearch
yum update squirro-storage-node
bash /opt/squirro/elasticsearch/update/migrate-storage-node.sh
# wait until migrate-storage-node.sh finished
yum update squirro-cluster-node-users
yum update squirro-*
monit monitor all

cd /lib/systemd/system
for service in $(ls sq*d.service); do echo "Stopping $service"; systemctl stop $service; done
yum update squirro-storage-node-users
yum update elasticsearch
yum update squirro-storage-node
systemctl daemon-reload
bash /opt/squirro/elasticsearch/update/migrate-storage-node.sh
# wait until migrate-storage-node.sh finished
yum update squirro-cluster-node-users
yum update squirro-*
for service in $(ls sq*d.service); do echo "Starting $service"; systemctl start $service; done

2. Upgrade Storage and Cluster Nodes when they are on different servers (and there is only one storage node and one cluster node)

On the one cluster node, shut down most of the Squirro services like so:

for service in `ls /etc/monit.d/sq*d | sed -e "s|^.*/||" | grep -v "sqclusterd"`; do monit stop $service; done
# wait for `monit summary` to indicate that all but 5 services are stopped

cd /lib/systemd/system
for service in $(ls sq*d.service); do echo "Stopping $service"; systemctl stop $service; done
# the output of following statement should indicate that all sq*d services are stopped:
for service in $(ls sq*d.service); do echo "Status of $service"; systemctl status $service; done

Upgrade the one storage nodes by running:

yum update squirro-storage-node-users
yum update elasticsearch
yum update squirro-storage-node
bash /opt/squirro/elasticsearch/update/migrate-storage-node.sh
# wait until update-storage-node.sh finished

yum update squirro-storage-node-users
yum update elasticsearch
yum update squirro-storage-node
systemctl daemon-reload
bash /opt/squirro/elasticsearch/update/migrate-storage-node.sh
# wait until migrate-storage-node.sh finished

Upgrade the one cluster node by running:

yum update squirro-cluster-node-users
yum update squirro-*
monit monitor all

yum update squirro-cluster-node-users
yum update squirro-*
cd /lib/systemd/system
for service in $(ls sq*d.service); do echo "Starting $service"; systemctl start $service; done
# wait for the following statement to indicate that all sq*d services are started
for service in $(ls sq*d.service); do echo "Status of $service"; systemctl status $service; done

3. Upgrade multi-node clusters (multiple Storage Nodes and/or multiple Cluster Nodes)

Upgrading clusters of Squirro nodes to release 2.5.2 is very involved. Please contact Squirro support for assistance.

On each cluster node, shut down most of the Squirro services like so:

for service in `ls /etc/monit.d/sq*d | sed -e "s|^.*/||" | grep -v "sqclusterd"`; do monit stop $service; done
# wait for `monit summary` to indicate that all but 5 services are stopped

cd /lib/systemd/system
for service in $(ls sq*d.service); do echo "Stopping $service"; systemctl stop $service; done
# wait for the following statement to indicate that all but 5 services are stopped
for service in $(ls sq*d.service); do echo "Status of $service"; systemctl status $service; done

Stop elasticsearch service on every storage nodes, check that no elasticsearch service running:

service elasticsearch stop
ps -ef | grep elasticsearch

systemctl stop elasticsearch
ps -ef | grep elasticsearch

Upgrade every storage nodes by running:

yum update squirro-storage-node-users
yum update elasticsearch

yum update squirro-storage-node-users
yum update elasticsearch

and then on every storage node:

yum update squirro-storage-node

systemctl daemon-reload
yum update squirro-storage-node

Migrate data from old elasticsearch template v7 index to new elasticsearch index template v8. Run this on ONLY ONE storage node.

bash /opt/squirro/elasticsearch/update/migrate-storage-node.sh
# wait until update-storage-node.sh finished

bash /opt/squirro/elasticsearch/update/migrate-storage-node.sh
# wait until migrate-storage-node.sh finished

New ES does not allow to run some plugins likes head or kopf anymore, therefore you need to start them on your local machine and use port forward to a storage node to connect  to ES. We recommend you to use an addon to see status of ES cluster, index and shards during upgrade when you have multiple nodes setup. For example if you use  cerebro:

Installation:

• Download from https://github.com/lmenezes/cerebro/releases
• Extract files
• Run bin/cerebro
• Access on http://localhost:9000

If you use port 9500 as port forward to ES node, then use: ssh -L 9500:storagehost:9200. After that you can connect cerebro to http://localhost:9500

Upgrade each cluster nodes by running:

yum update squirro-cluster-node-users
yum update squirro-python-squirro.service.cluster

yum update squirro-*
monit monitor all

yum update squirro-cluster-node-users
yum update squirro-*
for service in $(ls sq*d.service); do echo "Starting $service"; systemctl start $service; done

4. Elasticsearch settings

• ElasticSearch memory is set in /etc/elasticsearch/jvm.options, not in /etc/sysconfig/elasticsearch anymore.
  
  
• To change number of shards and replicas for your cluster, you have to update settings in the templates before creating index:

 # edit /etc/elasticsearch/templates/squirro_v8.json, change values:

"settings": {
   "index": {
       "number_of_shards": 6,
       "number_of_replicas": 0,
       ...
   }
}

$ ./etc/elasticsearch/templates/ensure_templates.sh
# check that templates is set correctly:
$ curl 'http://localhost:9200/_template?pretty'

# after finishing running 
# check that templates is set correctly:
$ curl 'http://localhost:9200/_template?pretty'