一， 基本環(huán)境介紹及基本環(huán)境配置
節(jié)點(diǎn)1： node1.xxxxxx.com 192.168.1.35 centos6.5_64 添加8g新硬盤
節(jié)點(diǎn)2： node1.xxxxxx.com 192.168.1.36 centos6.5_64 添加8g新硬盤
vip 192.168.1.39
節(jié)點(diǎn)1與節(jié)點(diǎn)2均需配置
修改主機(jī)名：
vim /etc/sysconfig/network
hostname=node1.xxxxxx.com配置hosts解析：
vim /etc/hosts
192.168.1.35 node1.xxxxxx.com node1
192.168.1.36 node2.xxxxxx.com node1同步系統(tǒng)時(shí)間：
ntpdate cn.pool.ntp.org關(guān)閉防火墻與selinux
service iptables stop
chkconfig iptables off
cat /etc/sysconfig/selinux
selinux=disabled以上配置在兩個(gè)節(jié)點(diǎn)都需要配置，配置完成之后重啟兩個(gè)節(jié)點(diǎn)
二:配置ssh互信
[root@node1～]#ssh-keygen -t rsa -b 1024
[root@node1～]#ssh-copy-id root@192.168.1.36
[root@node2～]#ssh-keygen -t rsa -b 1024
[root@node2～]#ssh-copy-id root@192.168.1.35三：drbd的安裝與配置（node1和node2執(zhí)行相同操作）
[root@node1～]#wget -c http://elrepo.org/linux/elrepo/el6/x86_64/rpms/drbd84-utils-8.4.2-1.el6.elrepo.x86_64.rpm
[root@node1～]#wget -c http://elrepo.org/linux/elrepo/el6/x86_64/rpms/kmod-drbd84-8.4.2-1.el6_3.elrepo.x86_64.rpm
[root@node1～]#rpm -ivh *.rpm獲取一個(gè)sha1值做為shared-secret
[root@node1～]#sha1sum /etc/drbd.conf
8a6c5f3c21b84c66049456d34b4c4980468bcfb3 /etc/drbd.conf創(chuàng)建并編輯資源配置文件:/etc/drbd.d/dbcluster.res
代碼如下:
[root@node1～]# vim /etc/drbd.d/dbcluster.res
resource dbcluster {
protocol c;
net {
cram-hmac-alg sha1;
shared-secret 8a6c5f3c21b84c66049456d34b4c4980468bcfb3;
after-sb-0pri discard-zero-changes;
after-sb-1pri discard-secondary;
after-sb-2pri disconnect;
rr-conflict disconnect;
}
device /dev/drbd0;
disk /dev/sdb1;
meta-disk internal;
on node1.xxxxxx.com {
address 192.168.1.35:7789;
}
on node2.xxxxxx.com {
address 192.168.1.36:7789;
}
}以上配置所用參數(shù)說明:
resource: 資源名稱
protocol: 使用協(xié)議”c”表示”同步的”,即收到遠(yuǎn)程的寫入確認(rèn)之后,則認(rèn)為寫入完成.
net: 兩個(gè)節(jié)點(diǎn)的sha1 key是一樣的
after-sb-0pri : “split brain”發(fā)生時(shí)且沒有數(shù)據(jù)變更,兩節(jié)點(diǎn)之間正常連接
after-sb-1pri : 如果有數(shù)據(jù)變更,則放棄輔設(shè)備數(shù)據(jù),并且從主設(shè)備同步
rr-conflict: 假如前面的設(shè)置不能應(yīng)用,并且drbd系統(tǒng)有角色沖突的話,系統(tǒng)自動(dòng)斷開節(jié)點(diǎn)間連接
meta-disk: meta data保存在同一個(gè)磁盤(sdb1)
on <node>: 組成集群的節(jié)點(diǎn)
將drbd配置拷貝到node機(jī)器:
[root@node1～]#scp /etc/drbd.d/dbcluster.res root@192.168.1.36:/etc/drbd.d/創(chuàng)建資源及文件系統(tǒng):
創(chuàng)建分區(qū)(未格式化過)
在node1和node2上創(chuàng)建lvm分區(qū)：
[root@node1～]#fdisk /dev/sdb
在node1和node2上給資源(dbcluster)創(chuàng)建meta data：
[root@node1～drbd]#drbdadm create-md dbcluster
激活資源（node1和node2都得查看）
– 首先確保drbd module已經(jīng)加載
查看是否加載:
# lsmod | grep drbd若未加載,則需加載:
# modprobe drbd
# lsmod | grep drbd
drbd 317261 0
libcrc32c 1246 1 drbd– 啟動(dòng)drbd后臺(tái)進(jìn)程:
[root@node1 drbd]# drbdadm up dbcluster
[root@node2 drbd]# drbdadm up dbcluster查看（node1和node2）drbd狀態(tài):
[root@node2 drbd]# /etc/init.d/drbd status
git-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by dag@build64r6, 2012-09-06 08:16:10
m:res cs ro ds p mounted fstype
0:dbcluster connected secondary/secondary inconsistent/inconsistent c從上面的信息可以看到,drbd服務(wù)已經(jīng)在兩臺(tái)機(jī)器上運(yùn)行,但任何一臺(tái)機(jī)器都不是主機(jī)器(“primary” host),因此無法訪問到資源(block device).
開始同步：
僅在主節(jié)點(diǎn)操作(這里為node1)
[root@node1 drbd]# drbdadm — –overwrite-data-of-peer primary dbcluster查看同步狀態(tài):
[root@node1 drbd.d]# cat /proc/drbd
version: 8.4.2 (api:1/proto:86-101)
git-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by dag@build64r6, 2012-09-06 08:16:10
0: cs:connected ro:primary/secondary ds:uptodate/uptodate c r—–
ns:8297248 nr:0 dw:0 dr:8297912 al:0 bm:507 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0上面的輸出結(jié)果的一些說明:
cs (connection state): 網(wǎng)絡(luò)連接狀態(tài)
ro (roles): 節(jié)點(diǎn)的角色(本節(jié)點(diǎn)的角色首先顯示)
ds (disk states):硬盤的狀態(tài)
復(fù)制協(xié)議: a, b or c(本配置是c)
看到drbd狀態(tài)為”cs:connected ro:primary/secondary ds:uptodate/uptodate”即表示同步結(jié)束.
也可以這樣查看drbd狀態(tài):
[root@centos193 drbd]# drbd-overview
0:dbcluster/0 connected secondary/primary uptodate/uptodate c r—–創(chuàng)建文件系統(tǒng)：
在主節(jié)點(diǎn)(node1)創(chuàng)建文件系統(tǒng):
[root@node1 drbd]# mkfs -t ext4 /dev/drbd0
mke2fs 1.41.12 (17-may-2010)
filesystem label=
os type: linux
block size=4096 (log=2)
…….
180 days, whichever comes first. use tune2fs -c or -i to override.注:沒必要在輔節(jié)點(diǎn)(node2)做同樣的操作,因?yàn)閐rbd會(huì)處理原始磁盤數(shù)據(jù)的同步.
另外,我們也不需要將這個(gè)drbd系統(tǒng)掛載到任何一臺(tái)機(jī)器(當(dāng)然安裝mysql的時(shí)候需要臨時(shí)掛載來安裝mysql),因?yàn)榧汗芾碥浖?huì)處理.還有要確保復(fù)制的文件系統(tǒng)僅僅掛載在active的主服務(wù)器上.
三：mysql的安裝
1,在node1和node2節(jié)點(diǎn)安裝mysql:
yum install mysql* -y2.node1和node2都操作停止mysql服務(wù)
[root@node1～]# service mysql stop
shutting down mysql. [ ok ]3.node1和node2都操作創(chuàng)建數(shù)據(jù)庫目錄并將該目錄權(quán)限屬主修改為mysql
[root@host1 /]# mkdir -p /mysql/data
[root@host1 /]# chown -r mysql:mysql /mysql4，關(guān)閉mysql臨時(shí)掛載drbd文件系統(tǒng)到主節(jié)點(diǎn)(node1)：
[root@node1 ~]# mount /dev/drbd0 /mysql/5.node1和node2都操作修改my.cnf文件修改
在[mysqld]下添加新的數(shù)據(jù)存放路徑
datadir=/mysql/data7.將默認(rèn)的數(shù)據(jù)路徑下的所有文件和目錄cp到新的目錄下（node2不用操作）
[root@host1 mysql]#cd /var/lib/mysql
[root@host1 mysql]#cp -r * /mysql/data/node1和node2都操作這里注意copy過去的目錄權(quán)限屬主需要修改為mysql,這里直接修改mysql目錄即可.
[root@host1 mysql]# chown -r mysql:mysql /mysql8.啟動(dòng)node1上的mysql進(jìn)行登陸測試
[root@host1 mysql]# mysql9.在節(jié)點(diǎn)node1卸載drbd文件系統(tǒng)
[root@node1 ~]# umount /var/lib/mysql_drbd
[root@node1 ~]# drbdadm secondary dbcluster將drbd文件系統(tǒng)掛載節(jié)點(diǎn)node2
[root@node2 ~]# drbdadm primary dbcluster
[root@node2 ~]# mount /dev/drbd0 /mysql/節(jié)點(diǎn)node2上配置mysql并測試
[root@node1 ~]# scp node2:/etc/my.cnf /etc/my.cnf
[root@node2 ~]# chown mysql /etc/my.cnf
[root@node2 ~]# chmod 644 /etc/my.cnf10. node2上做mysql登陸測試
[root@node2 ~]# mysql11.在node2上卸載drbd文件系統(tǒng),交由集群管理軟件pacemaker來管理
[root@node2～]# umount /var/lib/mysql_drbd
[root@node2～]# drbdadm secondary dbcluster
[root@node2～]# drbd-overview
0:dbcluster/0 connected secondary/secondary uptodate/uptodate c r—–
[root@node2～]#四：corosync和pacemaker的安裝配置（node1和node2都需安裝）
安裝pacemaker必須依賴:
[root@node1～]#yum -y install automake autoconf libtool-ltdl-devel pkgconfig python glib2-devel libxml2-devel libxslt-devel python-devel gcc-c++ bzip2-devel gnutls-devel pam-devel libqb-devel安裝cluster stack依賴:
[root@node1～]yum -y install clusterlib-devel corosynclib-devel安裝pacemaker可選依賴:
[root@node1～]yum -y install ncurses-devel openssl-devel cluster-glue-libs-devel docbook-style-xslpacemaker安裝:
[root@node1～]yum -y install pacemakercrmsh安裝:
[root@node1～]wget http://download.opensuse.org/repositories/network:/ha-clustering:/stable/centos_centos-6/network:ha-clustering:stable.repo
[root@node1～]yum -y install crmsh1，配置corosync
corosync key
– 生成節(jié)點(diǎn)間安全通信的key:
[root@node1～]# corosync-keygen– 將authkey拷貝到node2節(jié)點(diǎn)(保持authkey的權(quán)限為400):
[root@node～]# scp /etc/corosync/authkey node2:/etc/corosync/2，[root@node1～]# cp /etc/corosync/corosync.conf.example /etc/corosync/corosync.conf
編輯/etc/corosync/corosync.conf:
代碼如下:
# please read the corosync.conf.5 manual page
compatibility: whitetank
aisexec {
user: root
group: root
}
totem {
version: 2
secauth: off
threads: 0
interface {
ringnumber: 0
bindnetaddr: 192.168.1.0
mcastaddr: 226.94.1.1
mcastport: 4000
ttl: 1
}
}
logging {
fileline: off
to_stderr: no
to_logfile: yes
to_syslog: yes
logfile: /var/log/cluster/corosync.log
debug: off
timestamp: on
logger_subsys {
subsys: amf
debug: off
}
}
amf {
mode: disabled
}– 創(chuàng)建并編輯/etc/corosync/service.d/pcmk,添加”pacemaker”服務(wù)
[root@node1～]# cat /etc/corosync/service.d/pcmk
service {
# load the pacemaker cluster resource manager
name: pacemaker
ver: 1
}將上面兩個(gè)配置文件拷貝到另一節(jié)點(diǎn)
[root@node1]# scp /etc/corosync/corosync.conf node2:/etc/corosync/corosync.conf
[root@node1]# scp /etc/corosync/service.d/pcmk node2:/etc/corosync/service.d/pcmk3，啟動(dòng)corosync和pacemaker
分別在兩個(gè)節(jié)點(diǎn)上啟動(dòng)corosync并檢查.
[root@node1]# /etc/init.d/corosync start
starting corosync cluster engine (corosync): [ ok ]
[root@node1～]# corosync-cfgtool -s
printing ring status.
local node id -1123964736
ring id 0
id = 192.168.1.189
status = ring 0 active with no faults
[root@node2]# /etc/init.d/corosync start
starting corosync cluster engine (corosync): [ ok ]
– 在兩節(jié)點(diǎn)上分別啟動(dòng)pacemaker:
[root@node1~]# /etc/init.d/pacemaker start
starting pacemaker cluster manager: [ ok ]
[root@node2~]# /etc/init.d/pacemaker start
starting pacemaker cluster manager:資源配置
配置資源及約束
配置默認(rèn)屬性
查看已存在的配置:
[root@node1 ~]# crm configure show
node node1.xxxxxx.com
node node2.xxxxxx.com
property $id=cib-bootstrap-options dc-version=1.1.8-7.el6-394e906 cluster-infrastructure=classic openais (with plugin) expected-quorum-votes=2禁止stonith錯(cuò)誤:
[root@node1 ~]# crm configure property stonith-enabled=false[root@node1 ~]# crm_verify -l
讓集群忽略quorum:
[root@node1~]# crm configure property no-quorum-policy=ignore防止資源在恢復(fù)之后移動(dòng):
[root@node1~]# crm configure rsc_defaults resource-stickiness=100設(shè)置操作的默認(rèn)超時(shí):
[root@node1~]# crm configure property default-action-timeout=180s設(shè)置默認(rèn)的啟動(dòng)失敗是否為致命的:
[root@node1~]# crm configure property start-failure-is-fatal=false配置drbd資源
– 配置之前先停止drbd:
[root@node1~]# /etc/init.d/drbd stop
[root@node1~]# /etc/init.d/drbd stop– 配置drbd資源:
[root@node1~]# crm configure
crm(live)configure# primitive p_drbd_mysql ocf:linbit:drbd params drbd_resource=dbcluster op monitor interval=15s op start timeout=240s op stop timeout=100s– 配置drbd資源主從關(guān)系(定義只有一個(gè)master節(jié)點(diǎn)):
crm(live)configure# ms ms_drbd_mysql p_drbd_mysql meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true– 配置文件系統(tǒng)資源,定義掛載點(diǎn)(mount point):
crm(live)configure# primitive p_fs_mysql ocf:heartbeat:filesystem params device=/dev/drbd0 directory=/var/lib/mysql_drbd/ fstype=ext4配置vip資源
crm(live)configure# primitive p_ip_mysql ocf:heartbeat:ipaddr2 params ip=192.168.1.39 cidr_netmask=24 op monitor interval=30s配置mysql資源
crm(live)configure# primitive p_mysql lsb:mysql op monitor interval=20s timeout=30s op start interval=0 timeout=180s op stop interval=0 timeout=240s組資源和約束
通過”組”確保drbd,mysql和vip是在同一個(gè)節(jié)點(diǎn)(master)并且確定資源的啟動(dòng)/停止順序.
啟動(dòng): p_fs_mysql–>p_ip_mysql->p_mysql
停止: p_mysql–>p_ip_mysql–>p_fs_mysql
crm(live)configure# group g_mysql p_fs_mysql p_ip_mysql p_mysql組group_mysql永遠(yuǎn)只在master節(jié)點(diǎn):
crm(live)configure# colocation c_mysql_on_drbd inf: g_mysql ms_drbd_mysql:mastermysql的啟動(dòng)永遠(yuǎn)是在drbd master之后:
crm(live)configure# order o_drbd_before_mysql inf: ms_drbd_mysql:promote g_mysql:start配置檢查和提交
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# quit查看集群狀態(tài)和failover測試
狀態(tài)查看:
[root@node1 mysql]# crm_mon -1rfailover測試:
將node1設(shè)置為standby狀態(tài)
[root@node1 ~]# crm node standby過幾分鐘查看集群狀態(tài)(若切換成功,則看到如下狀態(tài)):
[root@node1 ~]# crm status
將node1恢復(fù)online狀態(tài):
[root@node1 mysql]# crm node online
[root@node1 mysql]# crm status