= Disclaimer =

The following comes with no warranty whatsoever. I am not responsible for any data loss or other issues that may arise from following these instructions. Please make backups of all files and test this thoroughly in your lab environment before using it in production.

= Info =

This document is a companion to the official documentation from Integrics at http://integrics.com/enswitch/guides/3.11/en/field/cdrs/

Some of the commands are borrowed from the above page and from other Integrics documentation.

The Enswitch version used is 3.11, and the servers run Ubuntu 12.04 64bit.

The servers used in this example are as follows:

*enswitchdb00 - Main active database server
*enswitchdb01 - Main standby database server
*enswitchcdrdb00 - Server where CDRs will be archived
*enswitchcdrdb01 - Server where CDRs will be archived

= OS install =

Load servers with the same OS as the current Enswitch database servers, in this example I use Ubuntu 12.04 64bit.

=== Update all OS packages: ===

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get autoremove
sudo init 6

Install an appropriate firewall, this is out of the scope of this document, but an example may be added later.

=== Install optional packages: ===

sudo apt-get install htop iotop bwm-ng tshark

= Database install and configuration =

=== Install MySQL on enswitchcdrdb00 and enswitchcdrdb01: ===

sudo apt-get install mysql-server

=== Configure master/master replication between enswitchcdrdb00 and enswitchcdrdb01: ===

Enable remote connections to MySQL on enswitchcdrdb00 and enswitchcdrdb01. In /etc/mysql.my.cnf, change the bind-address variable to 0.0.0.0.

Add the following configuration options to /etc/mysql/my.cnf under [mysqld] to enable binary logging on enswitchcdrdb00 and enswitchcdrdb01:

log_bin = /var/lib/mysql/mysql-bin.log

Add the following configuration options to /etc/mysql/my.cnf under [mysqld] to limit replication to only the enswitch database:

replicate-do-db = enswitch

Add the following configuration options to /etc/mysql/my.cnf under [mysqld] on enswitchcdrdb00:

server-id = 10

Add the following configuration options to /etc/mysql/my.cnf under [mysqld] on enswitchcdrdb01:

server-id = 11

Add the following configuration options to /etc/mysql/my.cnf under [mysqld] to enable a single file per table:

innodb_file_per_table

Restart MySQL:

sudo service mysql restart

Configure replication

Add a replicate user on enswitchcdrdb00:

grant super, replication client, replication slave, reload on *.* to replicate@10.1.0.89 identified by 'PASSWORD';

Add a replicate user on enswitchcdrdb01:

grant super, replication client, replication slave, reload on *.* to replicate@10.1.0.88 identified by 'PASSWORD';

On enswitchcdrdb00, display the current log file and position:

show master status\G

Insert the appropriate values instead of LOGFILE and POSITION below, and then run the command in mysql on enswitchcdrdb01:

change master to master_host='10.1.0.88', master_user='replicate', master_password='PASSWORD', master_log_file='LOGFILE', master_log_pos=POSITION;

Start the slave process on enswitchcdrdb01:

slave start;

On enswitchcdrdb01, display the current log file and position:

show master status\G

Insert the appropriate values instead of LOGFILE and POSITION below, and then run the command in mysql on enswitchcdrdb00:

change master to master_host='10.1.0.89', master_user='replicate', master_password='PASSWORD', master_log_file='LOGFILE', master_log_pos=POSITION;

Start the slave process on enswitchcdrdb00:

slave start;

= Install and configure Heartbeat =

=== Add entries to /etc/hosts for each server on enswitchcdrdb00 and enswitchcdrdb01: ===

10.1.0.88 enswitchcdrdb00
10.1.0.89 enswitchcdrdb01

=== Install heartbeat: ===

sudo apt-get install heartbeat

=== Configure heartbeat: ===

Create /etc/ha.d/haresources:

On enswitchcdrdb00 and enswitchcdrdb01:

enswitchcdrdb00 IPaddr::10.1.0.93/26/eth0

Create /etc/ha.d/ha.cf:

On enswitchcdrdb00:

debug 1
debugfile /var/log/heartbeat_debug.log
logfile /var/log/heartbeat.log
logfacility local0

keepalive 1
deadtime 10
warntime 5
initdead 60

udpport 700
bcast eth0
ucast eth0 10.1.0.89
ping 10.1.0.65

auto_failback off
node enswitchcdrdb00
node enswitchcdrdb01
respawn hacluster /usr/lib/heartbeat/ipfail

On enswitchcdrdb01:

debug 1
debugfile /var/log/heartbeat_debug.log
logfile /var/log/heartbeat.log
logfacility local0

keepalive 1
deadtime 10
warntime 5
initdead 60

udpport 700
bcast eth0
ucast eth0 10.1.0.88
ping 10.1.0.65

auto_failback off
node enswitchcdrdb00
node enswitchcdrdb01
respawn hacluster /usr/lib/heartbeat/ipfail

Create /etc/ha.d/authkeys on enswitchcdrdb00 and enswitchcdrdb01:

auth 2
1 crc
2 sha1 secret
3 md5 dhcp

Change permissions on /etc/ha.d/authkeys

sudo chmod 600 /etc/ha.d/authkeys

Add firewall rule allow heartbeat. This goes in /etc/firewall.sh anywhere after the "iptables -A INPUT -i lo -j ACCEPT" line and before "iptables -A INPUT -j LOG":

On enswitchcdrdb00:

iptables -t filter -A INPUT -p udp --dport 700 -s 10.1.0.89 -m comment --comment "Allow heartbeat from enswitchcdrdb01" -j ACCEPT

On enswitchcdrdb01:

iptables -t filter -A INPUT -p udp --dport 700 -s 10.1.0.88 -m comment --comment "Allow heartbeat from enswitchcdrdb00" -j ACCEPT

Apply new firewall rules:

sudo sh /etc/firewall.sh

=== Start Heartbeat: ===

Start heartbeat on enswitchcdrdb00:
sudo service heartbeat start

Wait until you see the 10.1.0.93 IP address on eth0:0 and drbd0 volume mounted at /mnt/drbd0, then start heartbeat on enswitchcdrdb01:
sudo service heartbeat start

You should now be able to test heartbeat by running hb_takeover on each box and having it take over the 10.1.0.93 IP and the drbd0 volume.
sudo /usr/share/heartbeat/hb_takeover

= Configure CDR archiving =

Make a backup of the current Enswitch database

Optionally, also backup the individual tables in the current database for easy retreival. NOTE, this should only be run on a test system, not on production since it will take a very long time.

CREATE TABLE cdrs_backup1 LIKE cdrs;
INSERT INTO cdrs_backup1 SELECT * FROM cdrs;
CREATE TABLE cdrcosts_backup1 LIKE cdrcosts;
INSERT INTO cdrcosts_backup1 SELECT * FROM cdrcosts;
CREATE TABLE cdrcost_taxes_backup1 LIKE cdrcost_taxes;
INSERT INTO cdrcost_taxes_backup1 SELECT * FROM cdrcost_taxes;

=== Create enswitch database on enswitchcdrdb00: ===

CREATE DATABASE enswitch;

=== Create tables: ===

Do 'SHOW CREATE TABLE' on enswitchcdb00 for each of the 3 cdr* tables and run these on enswitchcdrdb00.

Create enswitchcdrsrw user on enswitchcdrdb00"

GRANT ALL ON enswitch.* to enswitchcdrsrw IDENTIFIED BY 'password';

=== Run enswitch_cdrs_archive_remote manually: ===

sudo su - enswitch -c "/opt/enswitch/current/bin/enswitch_cdrs_archive_remote 365 debug"

=== Cron configuration ===

Disable the original enswitch_cdrs_archive if it is currently in use.

= Configure web servers =

=== Instruct web interface to use the archive CDR database ===

Add the following to /etc/enswitch/databases.conf on the web servers:

delete/cdrs/archive, 1, 100, mysql, 10.1.0.93, 3306, enswitch, enswitchcdrsrw, PASSWORD, 1
insert/cdrs/archive, 1, 100, mysql, 10.1.0.93, 3306, enswitch, enswitchcdrsrw, PASSWORD, 1
select/cdrs/archive, 1, 100, mysql, 10.1.0.93, 3306, enswitch, enswitchcdrsrw, PASSWORD, 1

=== Restart apache ===

sudo service apache2 restart

= Configure roles =

In order to allow non System Owner users to search archived CDRs, set the "Call history (archived)" to "Yes" under the appropriate roles. It would be best to set this for only roles where it is absolutely necessary.

= Configure backups of archived CDRs =

CDRs archived on a remote server will no longer be backed up by the standard Enswitch backup script.

=== backup script ===

This script will make a backup for each day, overwriting the last.

Create /usr/local/sbin/mysql-backup.sh withthe following contents:

#!/bin/sh

DAY=`date +%a`
HOSTNAME=`hostname`
MYSQL_USERNAME=$1
MYSQL_PASSWORD=$2
BACKUP_PATH=$3
COMPRESS_LEVEL=$4

rm -f $BACKUP_PATH/mysql_backup-$HOSTNAME-$DAY.sql*

mysqldump -u $MYSQL_USERNAME --password=$MYSQL_PASSWORD --all-databases --skip-lock-tables --single-transaction > $BACKUP_PATH/mysql_backup-$HOSTNAME-$DAY.sql

if [ $COMPRESS_LEVEL -gt 0 ] && [ $COMPRESS_LEVEL -lt 10 ]
then
xz -$COMPRESS_LEVEL $BACKUP_PATH/mysql_backup-$HOSTNAME-$DAY.sql
fi

Change permissions on /usr/local/sbin/mysql-backup.sh

sudo chmod +x /usr/local/sbin/mysql-backup.sh

Add cron entry:

echo "1 0 * * * root /usr/local/bin/mysql-backup.sh root PASSWORD /root/mysqlbackups 3" | sudo tee /etc/cron.d/mysql-backup

Since the cron file will contain a MySQL password, make it readable only by root

sudo chmod 700 /etc/cron.d/mysql-backup

= References =

https://integrics.com/enswitch/guides/3.11/en/field/cdrs/

https://integrics.com/enswitch/guides/3.11/en/field/install/mysql/replication/

Enswitch storage on NFS with DRBD and Heartbeat

2015-07-17T16:18:32Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-07-17T16:17:26Z

Danthony: /* Server configuration */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:34:51Z

Danthony: /* Kamailio->OpenSIPS: */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:34:37Z

Danthony: /* Kamailio->OpenSIPS: */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:34:25Z

Danthony: /* Kamailio->OpenSIPS: */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:33:49Z

Danthony: /* OpenSIPS->Kamailio: */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:33:28Z

Danthony: /* OpenSIPS->Kamailio: */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:33:10Z

Danthony: /* OpenSIPS->Kamailio: */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:32:15Z

Danthony: /* Overview */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:32:02Z

Danthony: /* OpenSIPS->Kamailio: */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:30:17Z

Danthony: /* The following allows the conversion from Kamailio back to OpenSIPS if needed: */

Migrating OpenSIPS location table to Kamailio format

2015-06-18T13:29:01Z

Danthony:

Enswitch storage on NFS with DRBD and Heartbeat

2015-06-17T21:20:03Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-22T18:31:09Z

Danthony: /* References */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-22T17:54:16Z

Danthony: /* References */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-22T17:54:04Z

Danthony: /* Cutover procedure */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-22T16:51:47Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-22T14:53:23Z

Danthony: /* Cutover procedure */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T19:08:44Z

Danthony: /* Cutover procedure */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T18:36:47Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T18:35:20Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T18:32:15Z

Danthony: /* Cutover procedure: */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T18:31:50Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T18:29:53Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T18:25:42Z

Danthony: /* Overview */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T18:24:59Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T18:03:19Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T16:20:14Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T16:04:29Z

Danthony: /* Cutover procedure: */

Enswitch storage on GlusterFS

2015-05-07T16:04:02Z

Danthony: /* Cutover procedure: */

== Disclaimer ==

The following comes with no warranty whatsoever. I am not responsible for any data loss or other issues that may arise from following these instructions. Please make backups of all files and test this thoroughly in your lab environment before using it in production.

== Overview ==

This document details the procedure for migrating a multi-machine Enswitch system from NFS storage to GlusterFS storage.

The procedure has been tested on Enswitch 3.11, but should work on most other versions. The GlusterFS servers run Ubuntu 14.04 64bit and the clients run Ubuntu 12.04 64bit.

The servers are as follows:

enswitchnfs0 - current active NFS server

enswitchnfs1 - current backup NFS server

enswitchstorage0 - New GlusterFS server 0

enswitchstorage1 - New GlusterFS server 1

The Enswitch subnet is 10.0.0.0/24

== Server configuration ==

Load enswitchstorage0 and enswitchstorage1 with Ubuntu 14.04 64bit. Make a partition for the OS and leave the rest of the disk empty for the GlusterFS volume. Do not create a swap partition, a swap file will be added later.

Update all OS packages on enswitchstorage0 and enswitchstorage1:

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get autoremove
sudo init 6

Upgrade 14.04 kernel to 3.16 to take advantage of new metadata checksum and free inode btree options in XFS:

sudo apt-get update
sudo apt-get install --install-recommends linux-generic-lts-utopic
sudo apt-get autoremove
sudo init 6

Create swap file:

sudo dd if=/dev/zero of=/swapfile0 bs=1M count=2048
sudo chmod 0600 /swapfile0
sudo mkswap /swapfile0

Add the following line to /etc/fstab:

/swapfile0 none swap sw 0 0

Enable swap file:

sudo swapon -a

Install additional software on enswitchstorage0 and enswitchstorage1 (optional):

sudo apt-get install htop iotop bwm-ng tshark

Add firewall rules on enswitchstorage0 and enswitchstorage1. The following can be used as the base for a firewall script:

iptables -F -m comment --comment "Clear all existing rules"
iptables -A INPUT -m state --state RELATED,ESTABLISHED -m comment --comment "Allow packets from related and established connections" -j ACCEPT
iptables -A INPUT -i lo -m comment --comment "Allow all on lo interface" -j ACCEPT
iptables -A INPUT -p tcp --dport 24007 -m state --state NEW -s 10.0.0.0/24 -m comment --comment "Allow GlusterFS Daemon from Enswitch subnet" -j ACCEPT
iptables -A INPUT -p tcp --dport 24008 -m state --state NEW -s 10.0.0.0/24 -m comment --comment "Allow GlusterFS Management from Enswitch subnet" -j ACCEPT
iptables -A INPUT -p tcp --dport 49152 -m state --state NEW -s 10.0.0.0/24 -m comment --comment "Allow connections to GlusterFS brick #1 from Enswitch subnet" -j ACCEPT
iptables -A INPUT -p tcp --dport 111 -m state --state NEW -s 10.0.0.0/24 -m comment --comment "Allow portmapper from Enswitch subnet" -j ACCEPT
iptables -A INPUT -p udp --dport 111 -s 10.0.0.0/24 -m comment --comment "Allow portmapper from Enswitch subnet" -j ACCEPT
iptables -A INPUT -m comment --comment "Log all unmatched packets" -j LOG
iptables -A INPUT -m comment --comment "Drop all unmatched packets" -j DROP

Configure apt source for GlusterFS 3.5:

echo "deb http://ppa.launchpad.net/gluster/glusterfs-3.5/ubuntu trusty main" | sudo tee /etc/apt/sources.list.d/glusterfs.list
echo "deb-src http://ppa.launchpad.net/gluster/glusterfs-3.5/ubuntu trusty main" | sudo tee -a /etc/apt/sources.list.d/glusterfs.list

Create a file named /tmp/glusterfs-repo.key contianing the following:

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: SKS 1.1.4
Comment: Hostname: keyserver.ubuntu.com

mQINBFPtYFcBEADcQMZ9aSR1ptbaEeq/8Bzu7kipaxVGh3Wcma4Lz+QPe0ofxRf+oYR22UVG
mJcPnVcGFbXJ50t8BAxwtQ/TSmGdQ93bl6LORAQBZ/ud1LTr2HKpaa0F1bwpi/TAgBWqP64H
u0LBGISc0G5m3/hn/bi6XxIIOzJ/L/vqLh1deVaDrYYWy5Cme8AuPtqOARKsefvVgwlpnbCt
k+QaE65vgl8MXiYCaOenT07GDCq1xb7hkoVlJS4bf6F3UMJVMVy4oEyYkRw4SP7ULeT1s4yr
BeDzbxhFaZRJFvGpvMW3AZxfrhX/5OpZSkQiFn5/2j4eJli4/MmptAAHpGr4tLA+s6mHmA9E
9c7wMfyFZe+wMhvangSDp09gSSZs00bqKSnYIJ/oGRjaxCllkw4SMfTOqv8l/GOxRs12yIcZ
D08SSmRpoyLffrl1zElyaixtAJRenphTZyq7eRLPyQl6qEDA1XtLs3ThK5/4fghMbe7MOHiM
B8MwL1RzLQkl/PU08vxfum9ki/m/LP5xpJopNHZs2L47RlX2+tq6FJWbDvQwOGoFTTnxmdDf
4EkMhlB4N+ujZw64pSMt3c08NShxty2UWpbSbc8/e7Ps4B7Lx6eq6AmqrcUChg8c9+PI2LUq
j6mDbc8jxpUslvjsLU05xnq6OLv4U//pUTUz6eI8FgFadVZcoQARAQABtBlMYXVuY2hwYWQg
UFBBIGZvciBHbHVzdGVyiQI4BBMBAgAiBQJT7WBXAhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIe
AQIXgAAKCRAT4Bt7P+hpqZ3LEACYYC4UjxwSHouV295Cxfwt9P32GcWJbFmLYtLHWVTt2vdN
/M9Xb02YgVLJm/nVy2vJhqcMowSW2jO503mLq672g5mHitnIq1lh4zXcHEvP79aDRQuvkgsL
EHjlk2NzYqdAsdRk3TgOLcK0SRM7Cwgwd/b/gVUtPYrX1hvQKrjGJM9VZFcCMX2RmGAS0ft3
QHzEAPZCgyamk0qB2eo8tLZYm42iMvq+ZSxGulhzi7gJkpv/wNdaP4E6o8o7KY3JIWMmxBn8
QZUKYMobze4PSBg4G4iG2ue9IrGCb8M1o+46aOSyEIc99bznF8Jrw7a8sBufVRjSZIE9A/oM
EtB1pTRDn9lwx/DyYbCV16DOsk6d5x4P8cqvgdaGzl7VNLvkwmMaCH0gRFIBr937rEUbeSJH
TqrVG0zXzSaUHEwXPZE0Lt2C9dEmMnT6nxC7FbJB1ATPDNx8kL7MvB4jl5HkjrD1W9Xu2y0d
zwAKlg5jvzwP46MJgvm+AYK808XhOhMZjWzzt5POeDcDhGhpRSfQtAhSnRkOtKS1drMCt27h
LZDEZfCp//aj7jvVL8FjamGEMfm91FLQa5LY7OoJaYoZlYUtthrXV6w5KHFjFYAKgA8tJzeb
Tvc1Q9avCo2G5qWNZq6TSLxHEMo/g4gu2aGRPRrKu9w2Ibosg4OqZ/YbXC8SjA==
=+Qna
-----END PGP PUBLIC KEY BLOCK-----

Import repository key:

sudo apt-key add /tmp/glusterfs-repo.key

Install GlusterFS packages on enswitchstorage0 and enswitchstorage1:

sudo apt-get update
sudo apt-get install glusterfs-server ntp xfsprogs

Download and install xfsprogs from a newer distribution. This is necessary to use the metadata checksum and free inode btree options:

cd /tmp
wget http://mirrors.kernel.org/ubuntu/pool/main/x/xfsprogs/xfsprogs_3.2.1ubuntu1_amd64.deb
sudo dpkg -i xfsprogs_3.2.1ubuntu1_amd64.deb

Add entries to /etc/hosts for each server on enswitchstorage0 and enswitchstorage1:

10.0.0.122 enswitchstorage0
10.0.0.123 enswitchstorage1

Create partition for the GlusterFS volume on enswitchstorage0 and enswitchstorage1, in this example we use /dev/sda2 as the glusterfs volume:

sudo fdisk
sudo mkdir -p /var/glusterfs/sda2

If your filesystem is on a standalone hard drive use the following options:

sudo mkfs.xfs -m crc=1,finobt=1 /dev/sda2

If your filesystem is on a raid device, add options for stripe size and stripe width. Stripe size is whatever was configured in the RAID controller when you create the volume, and stripe with is the number of data disks. See http://xfs.org/index.php/XFS_FAQ#Q:_How_to_calculate_the_correct_sunit.2Cswidth_values_for_optimal_performance for more information. In this example we are using a RAID 10 volume on 6 disks with a 64K stripe size:

sudo mkfs.xfs -m crc=1,finobt=1 -d su=64k,sw=3 /dev/sda2

Mount the GlusterFS partition on enswitchstorage0 and enswitchstorage1.

If the filesystem is smaller than 1TB add the following to /etc/fstab:

/dev/sda2 /var/glusterfs/sda2 xfs defaults 0 0

If the filesystem is 1TB or larger, use the inode64 option instead. This allows inodes to be placed above 1TB which will help avoid issues. See http://xfs.org/index.php/XFS_FAQ#Q:_What_is_the_inode64_mount_option_for.3F for a more in depth explanation:

/dev/sda2 /var/glusterfs/sda2 xfs inode64 0 0

Mount the partition manually:

sudo mount /var/glusterfs/sda2

Create directory for enswitch volume:

sudo mkdir /var/glusterfs/sda2/enswitch

Initialize the peer from enswitchstorage0 This command should return "peer probe: success":

sudo gluster peer probe enswitchstorage1

Check the peer status:
sudo gluster peer status

You should see this:

Number of Peers: 1

Hostname: enswitchstorage1
Port: 24007
Uuid: 2eed9049-4f5c-4e14-8d49-8935df95c9fe
State: Peer in Cluster (Connected)

Create GlusterFS volume, the only needs to be run on enswitchstorage0:

sudo gluster volume create enswitch replica 2 transport tcp enswitchstorage0:/var/glusterfs/sda2/enswitch enswitchstorage1:/var/glusterfs/sda2/enswitch

You should see the following if the creation was successful:

volume create: enswitch: success: please start the volume to access data

Start the volume:

sudo gluster volume start enswitch

Look at volume info:

sudo gluster volume info

Volume Name: enswitch
Type: Replicate
Volume ID: 3276be02-515f-41b1-ab22-a766724c8e64
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: enswitchstorage0:/var/glusterfs/sda2/enswitch
Brick2: enswitchstorage1:/var/glusterfs/sda2/enswitch

Configure settings for better performance:

sudo gluster volume set enswitch performance.cache-size 1GB

Renumber libuuid user and group to free up uid 100 and gid 101. Then change the UID/GID of the libuuid files to match their new groups:

sudo chown libuuid:libuuid /usr/sbin/uuidd
sudo chown libuuid:libuuid /var/lib/libuuid

Add Enswitch user and group. There is no Enswitch code on these boxes, but this will make the file ownership show "enswitch:enswitch":

sudo adduser --system --group --no-create-home --home /var/lib/enswitch/home --disabled-password enswitch

Create local mountpoint on enswitchstorage0 and enswitchstorage1 so that the shared files can be accessed locally. NOTE: do not modify any files directly under /var/glusterfs/sda2/enswitch, as this will cause corruption:

sudo mkdir /var/lib/enswitch

Add the following line to /etc/fstab on enswitchstorage0 and enswitchstorage1:

enswitchstorage0:/enswitch /var/lib/enswitch glusterfs direct-io-mode=disable,backupvolfile-server=enswitchstorage1,_netdev 0 0

Mount /var/lib/enswitch on enswitchstorage0 and enswitchstorage1:

sudo mount /var/lib/enswitch
sudo chown enswitch:enswitch /var/lib/enswitch
sudo chmod 775 /var/lib/enswitch
sudo chmod g+s /var/lib/enswitch

== Client install ==

Configure apt source for GlusterFS 3.5

echo "deb http://ppa.launchpad.net/gluster/glusterfs-3.5/ubuntu precise main" | sudo tee /etc/apt/sources.list.d/glusterfs.list
echo "deb-src http://ppa.launchpad.net/gluster/glusterfs-3.5/ubuntu precise main" | sudo tee -a /etc/apt/sources.list.d/glusterfs.list

Create a file named /tmp/glusterfs-repo.key containing the following:

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: SKS 1.1.4
Comment: Hostname: keyserver.ubuntu.com

mQINBFPtYFcBEADcQMZ9aSR1ptbaEeq/8Bzu7kipaxVGh3Wcma4Lz+QPe0ofxRf+oYR22UVG
mJcPnVcGFbXJ50t8BAxwtQ/TSmGdQ93bl6LORAQBZ/ud1LTr2HKpaa0F1bwpi/TAgBWqP64H
u0LBGISc0G5m3/hn/bi6XxIIOzJ/L/vqLh1deVaDrYYWy5Cme8AuPtqOARKsefvVgwlpnbCt
k+QaE65vgl8MXiYCaOenT07GDCq1xb7hkoVlJS4bf6F3UMJVMVy4oEyYkRw4SP7ULeT1s4yr
BeDzbxhFaZRJFvGpvMW3AZxfrhX/5OpZSkQiFn5/2j4eJli4/MmptAAHpGr4tLA+s6mHmA9E
9c7wMfyFZe+wMhvangSDp09gSSZs00bqKSnYIJ/oGRjaxCllkw4SMfTOqv8l/GOxRs12yIcZ
D08SSmRpoyLffrl1zElyaixtAJRenphTZyq7eRLPyQl6qEDA1XtLs3ThK5/4fghMbe7MOHiM
B8MwL1RzLQkl/PU08vxfum9ki/m/LP5xpJopNHZs2L47RlX2+tq6FJWbDvQwOGoFTTnxmdDf
4EkMhlB4N+ujZw64pSMt3c08NShxty2UWpbSbc8/e7Ps4B7Lx6eq6AmqrcUChg8c9+PI2LUq
j6mDbc8jxpUslvjsLU05xnq6OLv4U//pUTUz6eI8FgFadVZcoQARAQABtBlMYXVuY2hwYWQg
UFBBIGZvciBHbHVzdGVyiQI4BBMBAgAiBQJT7WBXAhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIe
AQIXgAAKCRAT4Bt7P+hpqZ3LEACYYC4UjxwSHouV295Cxfwt9P32GcWJbFmLYtLHWVTt2vdN
/M9Xb02YgVLJm/nVy2vJhqcMowSW2jO503mLq672g5mHitnIq1lh4zXcHEvP79aDRQuvkgsL
EHjlk2NzYqdAsdRk3TgOLcK0SRM7Cwgwd/b/gVUtPYrX1hvQKrjGJM9VZFcCMX2RmGAS0ft3
QHzEAPZCgyamk0qB2eo8tLZYm42iMvq+ZSxGulhzi7gJkpv/wNdaP4E6o8o7KY3JIWMmxBn8
QZUKYMobze4PSBg4G4iG2ue9IrGCb8M1o+46aOSyEIc99bznF8Jrw7a8sBufVRjSZIE9A/oM
EtB1pTRDn9lwx/DyYbCV16DOsk6d5x4P8cqvgdaGzl7VNLvkwmMaCH0gRFIBr937rEUbeSJH
TqrVG0zXzSaUHEwXPZE0Lt2C9dEmMnT6nxC7FbJB1ATPDNx8kL7MvB4jl5HkjrD1W9Xu2y0d
zwAKlg5jvzwP46MJgvm+AYK808XhOhMZjWzzt5POeDcDhGhpRSfQtAhSnRkOtKS1drMCt27h
LZDEZfCp//aj7jvVL8FjamGEMfm91FLQa5LY7OoJaYoZlYUtthrXV6w5KHFjFYAKgA8tJzeb
Tvc1Q9avCo2G5qWNZq6TSLxHEMo/g4gu2aGRPRrKu9w2Ibosg4OqZ/YbXC8SjA==
=+Qna
-----END PGP PUBLIC KEY BLOCK-----

Import repository key:

sudo apt-key add /tmp/glusterfs-repo.key

Install GlusterFS client packages:

sudo apt-get update
sudo apt-get install glusterfs-client ntp

== Cutover procedure: ==

On one of the current NFS boxes, mount the GlusterFS volume as /var/lib/enswitch2/ and rsync data:

sudo mkdir /var/lib/enswitch2
sudo mount -o direct-io-mode=disable,backupvolfile-server=enswitchstorage1 enswitchstorage0:/enswitch /var/lib/enswitch2
sudo rsync -av --delete /var/lib/enswitch/ /var/lib/enswitch2/

Unmount /var/lib/enswitch on all servers

sudo umount /var/lib/enswitch

Rsync data one more time on old NFS server:

sudo rsync -av --delete /var/lib/enswitch/ /var/lib/enswitch2/

Add the following line to /etc/fstab on all other servers (commenting out the current line for the /var/lib/enswitch nfs mount):

enswitchstorage0:/enswitch /var/lib/enswitch glusterfs direct-io-mode=disable,backupvolfile-server=enswitchstorage1,_netdev 0 0

Mount new GlusterFS volume on all Enswitch servers:

sudo mount /var/lib/enswitch

Restart enswitch on all servers

sudo enswitch restart

== References: ==

http://www.server-world.info/en/note?os=Ubuntu_14.04&p=glusterfs&f=2

http://www.jamescoyle.net/how-to/457-glusterfs-firewall-rules

http://www.jamescoyle.net/how-to/351-share-glusterfs-volume-to-a-single-ip-address

https://launchpad.net/~gluster

https://www.howtoforge.com/creating-an-nfs-like-standalone-storage-server-with-glusterfs-3.2.x-on-ubuntu-12.10

http://www.jamescoyle.net/how-to/559-glusterfs-performance-tuning

http://toruonu.blogspot.com/2012/12/xfs-vs-ext4.html

http://joejulian.name/blog/glusterfs-bit-by-ext4-structure-change/

http://xfs.org/index.php/XFS_FAQ

https://www.digitalocean.com/community/tutorials/how-to-create-a-redundant-storage-pool-using-glusterfs-on-ubuntu-servers

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T16:03:08Z

Danthony:

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:59:49Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:54:05Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:49:46Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:49:09Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:48:51Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:47:35Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:46:36Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:46:25Z

Danthony: /* Server configuration */

Enswitch storage on NFS with DRBD and Heartbeat

2015-05-07T15:45:00Z

Danthony: /* Server configuration */