這篇文章主要講解了“Ceph怎么添加刪除監視器”，文中的講解內容簡(jiǎn)單清晰，易于學(xué)習與理解，下面請大家跟著(zhù)小編的思路慢慢深入，一起來(lái)研究和學(xué)習“Ceph怎么添加刪除監視器”吧！
已有三節點(diǎn)ceph集群，有3個(gè)mon，現在再添加一個(gè)mon
# ceph -s
cluster 520d715f-adb5-4a6a-afb2-dcf586308166
health HEALTH_OK
monmap e3: 3 mons at {hadoop001=10.10.1.32:6789/0,hadoop002=10.10.1.33:6789/0,hadoop003=10.10.1.34:6789/0}
election epoch 1850, quorum 0,1,2hadoop001,hadoop002,hadoop003
osdmap e127: 4 osds: 4 up, 4 in
flags sortbitwise
pgmap v22405: 64 pgs, 1 pools, 0 bytes data, 0 objects
145 MB used, 334 GB / 334 GB avail
64 active+clean要添加新的mon節點(diǎn)，那新節點(diǎn)的系統環(huán)境也需要配置與原有環(huán)境一致，這里只簡(jiǎn)單列下需要配置的列表，不多做贅述：
主機名、/etc/hosts、ssh互信、防火墻、時(shí)間同步、Selinux、最大進(jìn)程數、文件句柄數、最大線(xiàn)程數、ceph的yum源
系統環(huán)境配置好后，在新增的mon節點(diǎn)上安裝ceph軟件
# yum install ceph
在原有mon節點(diǎn)上使用ceph-deploy直接創(chuàng )建新的mon
注意：配置文件中需要配置public_network,否則可能會(huì )添加失敗
# ceph-deploy mon create hadoop004
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.34): /usr/bin/ceph-deploy mon create hadoop004
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x2872cb0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] mon : ['hadoop004']
[ceph_deploy.cli][INFO ] func : <function mon at 0x27ff758>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] keyrings : None
[ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts hadoop004
[ceph_deploy.mon][DEBUG ] detecting platform for host hadoop004 ...
[hadoop004][DEBUG ] connected to host: hadoop004
[hadoop004][DEBUG ] detect platform information from remote host
[hadoop004][DEBUG ] detect machine type
[hadoop004][DEBUG ] find the location of an executable
[ceph_deploy.mon][INFO ] distro info: CentOS Linux 7.2.1511 Core
[hadoop004][DEBUG ] determining if provided host has same hostname in remote
[hadoop004][DEBUG ] get remote short hostname
[hadoop004][DEBUG ] deploying mon to hadoop004
[hadoop004][DEBUG ] get remote short hostname
[hadoop004][DEBUG ] remote hostname: hadoop004
[hadoop004][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[hadoop004][DEBUG ] create the mon path if it does not exist
[hadoop004][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-hadoop004/done
[hadoop004][DEBUG ] done path does not exist: /var/lib/ceph/mon/ceph-hadoop004/done
[hadoop004][INFO ] creating keyring file: /var/lib/ceph/tmp/ceph-hadoop004.mon.keyring
[hadoop004][DEBUG ] create the monitor keyring file
[hadoop004][INFO ] Running command: ceph-mon --cluster ceph --mkfs -i hadoop004 --keyring /var/lib/ceph/tmp/ceph-hadoop004.mon.keyring --setuser 167 --setgroup 167
[hadoop004][DEBUG ] ceph-mon: renaming mon.noname-d 10.10.1.36:6789/0 to mon.hadoop004
[hadoop004][DEBUG ] ceph-mon: set fsid to 520d715f-adb5-4a6a-afb2-dcf586308166
[hadoop004][DEBUG ] ceph-mon: created monfs at /var/lib/ceph/mon/ceph-hadoop004 for mon.hadoop004
[hadoop004][INFO ] unlinking keyring file /var/lib/ceph/tmp/ceph-hadoop004.mon.keyring
[hadoop004][DEBUG ] create a done file to avoid re-doing the mon deployment
[hadoop004][DEBUG ] create the init path if it does not exist
[hadoop004][INFO ] Running command: systemctl enable ceph.target
[hadoop004][INFO ] Running command: systemctl enable ceph-mon@hadoop004
[hadoop004][INFO ] Running command: systemctl start ceph-mon@hadoop004
[hadoop004][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.hadoop004.asok mon_status
[hadoop004][DEBUG ] ********************************************************************************
[hadoop004][DEBUG ] status for monitor: mon.hadoop004
[hadoop004][DEBUG ] {
[hadoop004][DEBUG ] "election_epoch": 0,
[hadoop004][DEBUG ] "extra_probe_peers": [
[hadoop004][DEBUG ] "10.10.1.32:6789/0",
[hadoop004][DEBUG ] "10.10.1.33:6789/0",
[hadoop004][DEBUG ] "10.10.1.34:6789/0"
[hadoop004][DEBUG ] ],
[hadoop004][DEBUG ] "monmap": {
[hadoop004][DEBUG ] "created": "2016-12-19 09:59:12.970500",
[hadoop004][DEBUG ] "epoch": 3,
[hadoop004][DEBUG ] "fsid": "520d715f-adb5-4a6a-afb2-dcf586308166",
[hadoop004][DEBUG ] "modified": "2017-08-02 17:22:40.247484",
[hadoop004][DEBUG ] "mons": [
[hadoop004][DEBUG ] {
[hadoop004][DEBUG ] "addr": "10.10.1.32:6789/0",
[hadoop004][DEBUG ] "name": "hadoop001",
[hadoop004][DEBUG ] "rank": 0
[hadoop004][DEBUG ] },
[hadoop004][DEBUG ] {
[hadoop004][DEBUG ] "addr": "10.10.1.33:6789/0",
[hadoop004][DEBUG ] "name": "hadoop002",
[hadoop004][DEBUG ] "rank": 1
[hadoop004][DEBUG ] },
[hadoop004][DEBUG ] {
[hadoop004][DEBUG ] "addr": "10.10.1.34:6789/0",
[hadoop004][DEBUG ] "name": "hadoop003",
[hadoop004][DEBUG ] "rank": 2
[hadoop004][DEBUG ] }
[hadoop004][DEBUG ] ]
[hadoop004][DEBUG ] },
[hadoop004][DEBUG ] "name": "hadoop004",
[hadoop004][DEBUG ] "outside_quorum": [],
[hadoop004][DEBUG ] "quorum": [],
[hadoop004][DEBUG ] "rank": -1,
[hadoop004][DEBUG ] "state": "probing",
[hadoop004][DEBUG ] "sync_provider": []
[hadoop004][DEBUG ] }
[hadoop004][DEBUG ] ********************************************************************************
[hadoop004][INFO ] monitor: mon.hadoop004 is currently at the state of probing
[hadoop004][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.hadoop004.asok mon_status
[hadoop004][WARNIN] monitor hadoop004 does not exist in monmap查看狀態(tài)，添加成功。
# ceph -s
cluster 520d715f-adb5-4a6a-afb2-dcf586308166
health HEALTH_OK
monmap e4: 4 mons at {hadoop001=10.10.1.32:6789/0,hadoop002=10.10.1.33:6789/0,hadoop003=10.10.1.34:6789/0,hadoop004=10.10.1.36:6789/0}
election epoch 1850, quorum 0,1,2,3 hadoop001,hadoop002,hadoop003,hadoop004
osdmap e127: 4 osds: 4 up, 4 in
flags sortbitwise
pgmap v22405: 64 pgs, 1 pools, 0 bytes data, 0 objects
145 MB used, 334 GB / 334 GB avail
64 active+clean添加完成后在ceph.conf中的mon_initial_members和mon_host參數中分別添加新mon節點(diǎn)的hostname和ip地址
# ceph-deploy mon destroy hadoop004 [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.34): /usr/bin/ceph-deploy mon destroy hadoop004 [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] overwrite_conf : False [ceph_deploy.cli][INFO ] subcommand : destroy [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x19f1cb0> [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] mon : ['hadoop004'] [ceph_deploy.cli][INFO ] func : <function mon at 0x197e758> [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.mon][DEBUG ] Removing mon from hadoop004 [hadoop004][DEBUG ] connected to host: hadoop004 [hadoop004][DEBUG ] detect platform information from remote host [hadoop004][DEBUG ] detect machine type [hadoop004][DEBUG ] find the location of an executable [hadoop004][DEBUG ] get remote short hostname [hadoop004][INFO ] Running command: ceph --cluster=ceph -n mon. -k /var/lib/ceph/mon/ceph-hadoop004/keyring mon remove hadoop004 [hadoop004][WARNIN] Error EINVAL: removing mon.hadoop004 at 10.10.1.36:6789/0, there will be 3 monitors [hadoop004][INFO ] polling the daemon to verify it stopped [hadoop004][INFO ] Running command: systemctl stop ceph-mon@hadoop004.service [hadoop004][INFO ] Running command: mkdir -p /var/lib/ceph/mon-removed [hadoop004][DEBUG ] move old monitor data
徹底清理,操作需慎重：
注意：這個(gè)會(huì )刪除mon節點(diǎn)hadoop004上所有的ceph數據，配置文件以及rpm包
# ceph-deploy purge hadoop004
如果覺(jué)得刪除的不干凈，可以再去hadoop004上刪除遺留目錄
# rm -rf /var/lib/ceph # rm -rf /var/run/ceph/*
上一章中已經(jīng)將hadoop004的mon清理干凈了。
hadoop004 上安裝軟件，創(chuàng )建mon目錄
[root@hadoop004 ~]# yum install ceph
hadoop001上將ceph.conf和客戶(hù)端密鑰拷貝到hadoop004的/etc/ceph目錄
# scp ceph.conf ceph.client.admin.keyring hadoop004:/etc/ceph/
hadoop004上： 獲取mon密鑰環(huán)
# mkdir dlw # cd dlw/ # ceph auth get mon. -o keying exported keyring for mon.
獲取監視器運行圖
# ceph mon getmap -o monmap got monmap epoch 5
創(chuàng )建監視器數據目錄
# ceph-mon -i hadoop004 --mkfs --monmap monmap --keyring keying ceph-mon: set fsid to 520d715f-adb5-4a6a-afb2-dcf586308166 ceph-mon: created monfs at /var/lib/ceph/mon/ceph-hadoop004 for mon.hadoop004
啟動(dòng)新監視器
# ceph-mon -i hadoop004 --public-addr 10.10.1.36:6789
檢查狀態(tài)
# ceph -s
cluster 520d715f-adb5-4a6a-afb2-dcf586308166
health HEALTH_OK
monmap e6: 4 mons at {hadoop001=10.10.1.32:6789/0,hadoop002=10.10.1.33:6789/0,hadoop003=10.10.1.34:6789/0,hadoop004=10.10.1.36:6789/0}
election epoch 1854, quorum 0,1,2,3 hadoop001,hadoop002,hadoop003,hadoop004
osdmap e127: 4 osds: 4 up, 4 in
flags sortbitwise
pgmap v22405: 64 pgs, 1 pools, 0 bytes data, 0 objects
145 MB used, 334 GB / 334 GB avail
64 active+clean發(fā)現集群已經(jīng)成功四個(gè)mon了，但是到這里并沒(méi)有完，ceph強在自我修復能力很強，總不能每次啟動(dòng)新的mon都要自己手動(dòng)執行ceph-mon。
添加ceph-mon@hadoop004服務(wù)
先找到剛剛啟動(dòng)的mon進(jìn)程，終止掉。
# ps -ef |grep ceph root 30514 1 0 18:25 pts/1 00:00:00 ceph-mon -i hadoop004 --public-addr 10.10.1.36:6789 root 30899 9739 0 18:30 pts/1 00:00:00 grep --color=auto ceph # kill 30514
在ceph.conf中的mon_initial_members和mon_host參數中分別添加新mon節點(diǎn)的hostname和ip地址
啟動(dòng)服務(wù)之前，需要修改mon數據目錄的權限為ceph
# cd /var/lib/ceph/mon # chown -R ceph:ceph ceph-hadoop004/
啟動(dòng)mon服務(wù)
# systemctl reset-failed ceph-mon@hadoop004.service # systemctl restart ceph-mon@`hostname` # systemctl enable ceph-mon@`hostname` # systemctl restart ceph-mon.target # systemctl status ceph-mon@`hostname` ● ceph-mon@hadoop004.service - Ceph cluster monitor daemon Loaded: loaded (/usr/lib/systemd/system/ceph-mon@.service; enabled; vendor preset: disabled) Active: active (running) since 三 2017-08-02 18:37:36 CHOST; 3s ago Main PID: 31115 (ceph-mon) CGroup: /system.slice/system-ceph\x2dmon.slice/ceph-mon@hadoop004.service └─31115 /usr/bin/ceph-mon -f --cluster ceph --id hadoop004 --setuser ceph --setgroup ceph 8月 02 18:37:36 hadoop004 systemd[1]: Started Ceph cluster monitor daemon. 8月 02 18:37:36 hadoop004 systemd[1]: Starting Ceph cluster monitor daemon... 8月 02 18:37:36 hadoop004 ceph-mon[31115]: starting mon.hadoop004 rank 3 at 10.10.1.36:6789/0 mon_data /var/lib/ceph/mon/ceph-hadoop004 fsid 520d715f-adb5-4a6a-afb2-dcf586308166
# ceph mon remove hadoop004 Error EINVAL: removing mon.hadoop004 at 10.10.1.36:6789/0, there will be 3 monitors
清理數據目錄，卸載軟件包
# rm -rf /var/lib/ceph # rm -rf /var/run/ceph/* # yum remove ceph
查看集群mon的選取情況 # ceph quorum_status -f json-pretty 獲取monmap # ceph-mon -i `hostname` --inject-monmap /opt/monmap 查看monmap # monmaptool --print /opt/monmap 在monmap中添加mon # monmaptool --add hadoop004 10.10.1.36:6789 在monmap中刪除mon # monmaptool /tmp/monmap --rm hadoop004 注入monmap，注入之前要停止所有的mon # systemctl stop ceph-mon@`hostname` # ceph-mon -i `hostname` --inject-monmap /opt/monmap
免責聲明：本站發(fā)布的內容（圖片、視頻和文字）以原創(chuàng )、來(lái)自本網(wǎng)站內容采集于網(wǎng)絡(luò )互聯(lián)網(wǎng)轉載等其它媒體和分享為主，內容觀(guān)點(diǎn)不代表本網(wǎng)站立場(chǎng)，如侵犯了原作者的版權，請告知一經(jīng)查實(shí)，將立刻刪除涉嫌侵權內容，聯(lián)系我們QQ:712375056，同時(shí)歡迎投稿傳遞力量。
Copyright ? 2009-2022 56dr.com. All Rights Reserved. 特網(wǎng)科技 特網(wǎng)云 版權所有 特網(wǎng)科技 粵ICP備16109289號
域名注冊服務(wù)機構:阿里云計算有限公司（萬(wàn)網(wǎng)） 域名服務(wù)機構：煙臺帝思普網(wǎng)絡(luò )科技有限公司（DNSPod） CDN服務(wù)：阿里云計算有限公司 百度云 中國互聯(lián)網(wǎng)舉報中心 增值電信業(yè)務(wù)經(jīng)營(yíng)許可證B2
建議您使用Chrome、Firefox、Edge、IE10及以上版本和360等主流瀏覽器瀏覽本網(wǎng)站
