Setup DRBD for master/master

At my job at OpenSky we have a common NFS share that is failed over to a slave nfs node if the master dies. It was setup so that the slave would just rsync from the master. This was done to save time. Not that I have some time, I decided to finally setup drbd on the two nodes to keep the files always in sync.

So below are my nodes with hostnames/ips and drive information

Node1
host: nfs1.domain.com
ip: 192.168.1.46
partition: /dev/sdb1

Node2
host: nfs2.domain.com
ip: 192.168.1.47
partition: /dev/sdb1

So once drbd is installed setup the following file

/etc/drbd.conf

To look something like this. You will want to change a few things in red. Put this file on both nodes.

global {
  usage-count yes;
}
common {
  protocol C;
}
resource nfs {
  meta-disk internal;
  device     /dev/drbd1;
  syncer {
    verify-alg sha1;
  }
  net {
    allow-two-primaries;
  }
  on nfs1.domain.com {
    disk       /dev/sdb1;
    address    192.168.1.46:7789;
  }
  on nfs2.domain.com {
    disk       /dev/sdb1;
    address    192.168.1.47:7789;
  }
}

I choose to use protocol C. You can see the following protocols below. Also the on nfs1.domain.com has to match the output of a hostname command

http://www.drbd.org/users-guide/s-replication-protocols.html

Now on both nodes you want to init the drbd partition. Make sure this is a blank partition. It will wipe out everything that is one it. I called my resource nfs. If you changed it the commands will be a bit different based on what name you gave your resource

drbdadm create-md nfs

You should get an output like

initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success

Now make sure drbd can be run on startup and start it up now

chkconfig drbd on
service drbd start

Once you start drbd on both nodes it will connect to both. If you just start one and not the other.. the other will wait for the second node to come alive.

You can now make a filesystem. Make sure it is a filesystem for clustering. I recommend ocfs2 but gfs2 also works.

> db.stats()
{
        "collections" : 19,
        "objects" : 145971,
        "dataSize" : 64785380,
        "storageSize" : 129544960,
        "numExtents" : 75,
        "indexes" : 69,
        "indexSize" : 25403392,
        "ok" : 1
}

Traceback (most recent call last):
  File "./get_mongodb_stats.py", line 68, in 
    main(sys.argv[1:])
  File "./get_mongodb_stats.py", line 39, in main
    get_stats(host, port)
  File "./get_mongodb_stats.py", line 57, in get_stats
    info = db.command("stats")
  File "/usr/lib/python2.6/site-packages/pymongo/database.py", line 306, in command
    (command, result["errmsg"]))

> db.stats  
function () {
    return this.runCommand({dbstats:1});
}

SOLR_CORES="core1,core2,core3"
declare -a CORES
CORES=(`echo $SOLR_CORES | tr ',' ' '`)       

for CORE in ${CORES[@]}
do
    stuff here
done