Wednesday, May 15, 2013

LVM snapshot backup and restore mongo DB sample

This is LVM snapshot validation test performed on jinlinux 


Seup lv 
lvcreate -n mongoData demo -L 10000M vg_jinlinux

[root@jinlinux ~]# lvscan
ACTIVE '/dev/vg_jinlinux/lv_root' [50.00 GiB] inherit
ACTIVE '/dev/vg_jinlinux/lv_home' [64.26 GiB] inherit
ACTIVE '/dev/vg_jinlinux/lv_swap' [3.73 GiB] inherit
ACTIVE Original '/dev/vg_jinlinux/demo' [11.09 GiB] inherit
ACTIVE Snapshot '/dev/vg_jinlinux/mongoSnap' [1.46 GiB] inherit
[root@jinlinux ~]# vgdisplay
--- Volume group ---
VG Name vg_jinlinux
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 28
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 5
Open LV 5
Max PV 0
Cur PV 1
Act PV 1
VG Size 148.52 GiB
PE Size 4.00 MiB
Total PE 38021
Alloc PE / Size 33420 / 130.55 GiB
Free PE / Size 4601 / 17.97 GiB
VG UUID BZRyNp-1wf6-VTa2-zOiD-IMyL-gbiS-5AUyFU

Create file system
mkfs ext4 -m 0 /dev/vg_jinlinux/demo

Create directory
[root@jinlinux ~]# mkdir /app/mongoData

Change Owner
chown mongod:mongod /app/mongoData

Mount lv 
[root@jinlinux mongod]# mount -t ext4 /dev/vg_jinlinux/demo /app/mongoData

edit /etc/mongod.conf Make changes to point to this new directory
    logpath=/var/log/mongo/mongod.log
    logappend=true
    fork = true
    port = 27021
    bind_ip = 127.0.0.1,10.128.213.66
    dbpath=/app/mongoData
    journal=true
    directoryperdb=true
    quota=true
    quotaFiles=10
    rest=true
    slowms=200
    oplogSize=100
    pidfilepath = /var/run/mongodb/mongod.pid

Start mongod 
mongod --smallfiles -f /etc/mongod.conf

Begin data inserting
invoke java program "dbMoverOracleToMongo.java" to insert less than 10000 rows into mongo.

During insert at db.usr_sdlc.count() shows 5550 do LVM snapshot
[root@jinlinux ~]# lvcreate -L1500M -s -n mongoSnap /dev/mapper/vg_jinlinux-demo

Let insert keep going for a while till reach db.usr_sdlc.count() show 8040

Validate snapshot created
[root@jinlinux mongod]# lvscan
ACTIVE '/dev/vg_jinlinux/lv_root' [50.00 GiB] inherit
ACTIVE '/dev/vg_jinlinux/lv_home' [64.26 GiB] inherit
ACTIVE '/dev/vg_jinlinux/lv_swap' [3.73 GiB] inherit
ACTIVE Original '/dev/vg_jinlinux/demo' [11.09 GiB] inherit
ACTIVE Snapshot '/dev/vg_jinlinux/mongoSnap' [1.46 GiB] inherit

shutdown mongod
mongo localhost:27021
use admin
db.shutdownServer();

create new directory to mount snapshot
mkdir /app/mongoSnap
chown mongod:mongod /app/mongoSnap

mount snapshot
[root@jinlinux mongod]# mount -t ext4 /dev/vg_jinlinux/mongoSnap /app/mongoSnap

Create new mongod.conf to bring up snapshot version of DB
[root@jinlinux mongod]# cat /home/mongod/mongodSnap.conf
logpath=/tmp/mongodSnap.log
logappend=true
fork = true
port = 27021
bind_ip = 127.0.0.1,10.128.213.66
dbpath=/app/mongoSnap
journal=true
directoryperdb=true
quota=true
quotaFiles=10
rest=true
slowms=200
oplogSize=100
pidfilepath = /tmp/mongod.pid

Bring up mongod with snapshot 
mongod --smallfiles -f /home/mongod/mongodSnap.conf

Show row count 
[root@jinlinux mongod]# mongo localhost:27021
MongoDB shell version: 2.4.3
connecting to: localhost:27021/test
> use admin
switched to db admin
> use test
switched to db test
> db.usr_sdlc.count();
5295
> use admin
switched to db admin
> db.shutdownServer();


Bring up original server and show row count 8040. The difference in counts shows the snapshot backup was done at the moment while active writing was happenning 

[root@jinlinux mongod]# mongo localhost:27021
MongoDB shell version: 2.4.3
connecting to: localhost:27021/test
> use test
switched to db test
> db.usr_sdlc.count();
8040
> db.shutdownServer();
shutdown command only works with the admin database; try 'use admin'
> use admin
switched to db admin
> db.shutdownServer();
Wed May 15 11:49:04.882 DBClientCursor::init call() failed
server should be down...
Wed May 15 11:49:04.883 trying reconnect to localhost:27021
Wed May 15 11:49:04.884 reconnect localhost:27021 ok
Wed May 15 11:49:04.884 Socket say send() errno:104 Connection reset by peer 127.0.0.1:27021




Unmount snapshot directory

[root@jinlinux app]# umount mongoSnap
[root@jinlinux app]# lvscan
ACTIVE '/dev/vg_jinlinux/lv_root' [50.00 GiB] inherit
ACTIVE '/dev/vg_jinlinux/lv_home' [64.26 GiB] inherit
ACTIVE '/dev/vg_jinlinux/lv_swap' [3.73 GiB] inherit
ACTIVE Original '/dev/vg_jinlinux/demo' [11.09 GiB] inherit
ACTIVE Snapshot '/dev/vg_jinlinux/mongoSnap' [1.46 GiB] inherit

Backup snapshot to different location
[root@jinlinux app]# dd if=/dev/vg_jinlinux/mongoSnap |gzip > /tmp/mongSnap.gz
23248896+0 records in
23248896+0 records out
11903434752 bytes (12 GB) copied, 217.663 s, 54.7 MB/s

[root@jinlinux app]# ls -alrt /tmp/mongSnap.gz
-rw-r--r--. 1 root root 958360718 May 15 13:55 /tmp/mongSnap.gz

Restore validation 
Create LV, make sure it is of the same size of bigger than the original lv
[root@jinlinux app]# lvcreate --size 12G --name snapRestore vg_jinlinux
Logical volume "snapRestore" created
[root@jinlinux app]# mkdir /app/snapRestore
[root@jinlinux app]# chown mongod:mongod /app/snapRestore

Restore the lv from backup
[root@jinlinux app]# gzip -d -c /tmp/mongSnap.gz |dd of=/dev/vg_jinlinux/snapRestore
23248896+0 records in
23248896+0 records out
11903434752 bytes (12 GB) copied, 513.228 s, 23.2 MB/s

Create mongodSnapRestore.conf like the following
-bash-4.1$ cat mongodSnapRestore.conf
logpath=/tmp/mongodSnapRestore.log
logappend=true
fork = true
port = 27021
bind_ip = 127.0.0.1,10.128.213.66
dbpath=/app/snapRestore
journal=true
directoryperdb=true
quota=true
quotaFiles=10
rest=true
slowms=200
oplogSize=100
pidfilepath = /tmp/mongodRestore.pid


Bring up Mongo database from /app/snapRestore location
-bash-4.1$ mongod --smallfiles -f /home/mongod/mongodSnapRestore.conf
forked process: 6178
all output going to: /tmp/mongodSnapRestore.log
child process started successfully, parent exiting

Show count
-bash-4.1$ mongo localhost:27021
MongoDB shell version: 2.4.3
connecting to: localhost:27021/test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
> use test
switched to db test
> db.usr_sdlc.count();
5295
>

No comments:

Post a Comment