Mogilefs雜記（1）
阿新 • • 發佈：2018-03-15
mogilefs 雜記 基礎Mogilefs
Mogilefs搭建
環境
master1:nginx反代節點
master2、3、4：成為mogilefs節點
master2:MySQL節點0、安裝依賴環境包
[root@master2 ~]# yum install -y perl-Sys-Syslog perl-Net-Netmask perl-IO-AIO [root@master3 ~]# yum install -y perl-Sys-Syslog perl-Net-Netmask perl-IO-AIO [root@master4 ~]# yum install -y perl-Sys-Syslog perl-Net-Netmask perl-IO-AIO yum install -y perl-core
1、mogilefs組件安裝
[root@master2 mogilefs]# ls MogileFS-Server-2.46-2.el6.noarch.rpm MogileFS-Utils-2.19-1.el6.noarch.rpm perl-MogileFS-Client-1.14-1.el6.noarch.rpm MogileFS-Server-mogilefsd-2.46-2.el6.noarch.rpm Perlbal-1.78-1.el6.noarch.rpm perl-Perlbal-1.78-1.el6.noarch.rpm MogileFS-Server-mogstored-2.46-2.el6.noarch.rpm perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm [root@master3 mogilefs]# yum install -y *.rpm 3、4節點同上
2、master搭建mysql
2.1 MariaDB配置
[root@master2 ~]# vim /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
innodb_file_per_table = 1
skip_name_resolve = 1
[root@master2 ~]# systemctl start mariadb.service
設置開機啟動mariadb：
[root@master2 ~]# systemctl enable mariadb.service2.2 mysql創建數據庫和用戶
創建遠程連接的root用戶：
MariaDB [(none)]> GRANT ALL ON *.* TO ‘root‘@‘10.201.106.%‘ IDENTIFIED BY ‘mageedu‘;
創建數據庫：
MariaDB [(none)]> CREATE DATABASE mogilefs;
創建普通賬號：
MariaDB [(none)]> GRANT ALL ON mogilefs.* TO ‘moguser‘@‘10.201.106.%‘ IDENTIFIED BY ‘mogpass‘;
MariaDB [(none)]> FLUSH PRIVILEGES;2.3 使用mogdbsetup使數據初始化
[root@master2 ~]# mogdbsetup --dbhost=10.201.106.132 --dbrootpass=‘magedu‘ --dbuser=‘moguser‘ --dbpass=‘mogpass‘
查看mogilefs數據庫的表格
MariaDB [mogilefs]> SHOW TABLES;
+----------------------+
| Tables_in_mogilefs |
+----------------------+
| checksum |
| class |
| device |
| domain |
| file |
| file_on |
| file_on_corrupt |
| file_to_delete |
| file_to_delete2 |
| file_to_delete_later |
| file_to_queue |
| file_to_replicate |
| fsck_log |
| host |
| server_settings |
| tempfile |
| unreachable_fids |
+----------------------+
17 rows in set (0.00 sec)3、配置mogilefs
3.1 master2：tracker配置
[root@master2 mogilefs]# vim mogilefsd.conf
# Enable daemon mode to work in background and use syslog
daemonize = 1
# Where to store the pid of the daemon (must be the same in the init script)
pidfile = /var/run/mogilefsd/mogilefsd.pid
# Database connection information
db_dsn = DBI:mysql:mogilefs:host=10.201.106.132
db_user = moguser
db_pass = mogpass
# IP:PORT to listen on for mogilefs client requests
listen = 0.0.0.0:70013.2
[root@master2 mogilefs]# rpm -qi MogileFS-Server-mogilefsd
Name : MogileFS-Server-mogilefsd
Version : 2.46
Release : 2.el6
Architecture: noarch
Install Date: Mon 16 Jan 2017 05:54:44 AM CST
Group : Applications/CPAN
Size : 535807
License : Artistic
Signature : (none)
Source RPM : MogileFS-Server-2.46-2.el6.src.rpm
Build Date : Tue 08 Oct 2013 06:48:24 PM CST
Build Host : www.magedu.com
Relocations : (not relocatable)
Packager : MageEdu <[email protected]>
Vendor : Alan Kasindorf <[email protected]>
Summary : MogileFS-Server-mogilefsd - Mogilefsd and related libraries.
Description :
[root@master2 mogilefsd]# service mogilefsd start4、mogilestorage
4.1 創建數據存儲目錄
[root@master2 ~]# mkdir -pv /data/mogilefs/dev1
mkdir: created directory ‘/data’
mkdir: created directory ‘/data/mogilefs’
mkdir: created directory ‘/data/mogilefs/dev1’
[root@master2 ~]# chown -R mogilefs.mogilefs /data/mogilefs/4.2 配置文件
[root@master2 ~]# vim /etc/mogilefs/mogstored.conf
maxconns = 10000
httplisten = 0.0.0.0:7500
mgmtlisten = 0.0.0.0:7501
docroot = /data/mogilefs/4.3 啟動服務
[root@master2 ~]# rpm -ql MogileFS-Server-mogstored
/etc/mogilefs/mogstored.conf
/etc/rc.d/init.d/mogstored
/usr/bin/mogautomount
/usr/bin/mogstored
/usr/share/man/man1/mogautomount.1.gz
/usr/share/man/man1/mogstored.1.gz
/usr/share/perl5/vendor_perl/Mogstored/ChildProcess
/usr/share/perl5/vendor_perl/Mogstored/ChildProcess.pm
/usr/share/perl5/vendor_perl/Mogstored/ChildProcess/DiskUsage.pm
/usr/share/perl5/vendor_perl/Mogstored/ChildProcess/IOStat.pm
/usr/share/perl5/vendor_perl/Mogstored/FIDStatter.pm
/usr/share/perl5/vendor_perl/Mogstored/HTTPServer
/usr/share/perl5/vendor_perl/Mogstored/HTTPServer.pm
/usr/share/perl5/vendor_perl/Mogstored/HTTPServer/Apache.pm
/usr/share/perl5/vendor_perl/Mogstored/HTTPServer/Lighttpd.pm
/usr/share/perl5/vendor_perl/Mogstored/HTTPServer/Nginx.pm
/usr/share/perl5/vendor_perl/Mogstored/HTTPServer/None.pm
/usr/share/perl5/vendor_perl/Mogstored/HTTPServer/Perlbal.pm
/usr/share/perl5/vendor_perl/Mogstored/SideChannelClient.pm
/usr/share/perl5/vendor_perl/Mogstored/SideChannelListener.pm
/usr/share/perl5/vendor_perl/Mogstored/TaskQueue.pm
[root@master2 ~]# vim /etc/rc.d/init.d/mogstored
#!/bin/bash
# Author: MageEdu <[email protected]>
# mogstored - Startup script for the MogileFS storage
#
# chkconfig: - 86 14
# description: MogileFS storage
# processname: mogstored
# config: /etc/mogilefs/mogstored.conf
# pidfile: /var/run/mogilefsd/mogstored.pid
# Source function library.
. /etc/rc.d/init.d/functions
# Path to the apachectl script, server binary, and short-form for messages.
pidfile=/var/run/mogilefs/mogstored.pid
lockfile=${LOCKFILE-/var/lock/subsys/mogstored}
RETVAL=0
configfile=‘/etc/mogilefs/mogstored.conf‘
prog=$(which mogstored)
echo -n $"Starting mogstored"
su - mogilefs -c "$prog -c $configfile --daemon" &> /dev/null
RETVAL=$?
echo
echo -n $"Starting mogstored"
su - mogilefs -c "$prog -c $configfile --daemon" &> /dev/null
RETVAL=$?
[ $RETVAL = 0 ] && success && touch ${lockfile} && echo $(pidof mogstored) > ${pidfile} || failure
echo
return $RETVAL
}
stop() {
echo -n $"Stopping mogstored"
netstat -nlp|grep "mogstored"|grep -v grep|awk ‘{print $7}‘|awk -F"/" ‘{print $1}‘|xargs kill -9
RETVAL=$?
[ $RETVAL = 0 ] && success && rm -f ${lockfile} ${pidfile} || failure
echo
}
[root@master2 ~]# systemctl daemon-reload
[root@master2 ~]# service mogstored start4.4 master3,4 storage配置
[root@master3 ~]# mkdir -pv /data/mogilefs/dev2
mkdir: created directory ‘/data’
mkdir: created directory ‘/data/mogilefs’
mkdir: created directory ‘/data/mogilefs/dev2’
[root@master3 ~]# chown -R mogilefs.mogilefs /data/mogilefs/
[root@master4 ~]# mkdir -pv /data/mogilefs/dev3
mkdir: created directory ‘/data’
mkdir: created directory ‘/data/mogilefs’
mkdir: created directory ‘/data/mogilefs/dev3’
[root@master4 ~]# chown -R mogilefs.mogilefs /data/mogilefs/
拷貝腳本：
[root@master2 mogilefs]# scp /etc/rc.d/init.d/mogstored master3:/etc/rc.d/init.d/
root@master3‘s password:
mogstored 100% 1456 1.4KB/s 00:00
[root@master2 mogilefs]# scp /etc/rc.d/init.d/mogstored master4:/etc/rc.d/init.d/
root@master4‘s password:
mogstored 100% 1456 1.4KB/s 00:00
[root@master2 mogilefs]#
[root@master2 mogilefs]# scp /etc/mogilefs/mogstored.conf master3:/etc/mogilefs/
root@master3‘s password:
mogstored.conf 100% 95 0.1KB/s 00:00
[root@master2 mogilefs]# scp /etc/mogilefs/mogstored.conf master4:/etc/mogilefs/
root@master4‘s password:
mogstored.conf 100% 95 0.1KB/s 00:00
[root@master2 mogilefs]#
啟動服務：
[root@master3 ~]# service mogstored start
Starting mogstored (via systemctl): [ OK ]
[root@master4 ~]# service mogstored start
Starting mogstored (via systemctl): [ OK ]5、tracker節點添加storage節點
5.1 檢查當前tracker節點信息
[root@master2 ~]# mogadm check
Checking trackers...
127.0.0.1:7001 ... OK
Checking hosts...
No devices found on tracker(s).
[root@master2 ~]# mogadm --trackers=10.201.106.132:7001 check
Checking trackers...
10.201.106.132:7001 ... OK
Checking hosts...
No devices found on tracker(s).5.2 添加主機
[root@master2 ~]# mogadm --trackers=10.201.106.132:7001 host add 10.201.106.132 --ip=10.201.106.132 --status=alive
[root@master2 ~]# mogadm host list
10.201.106.132 [1]: alive
IP: 10.201.106.132:7500
添加另外兩個節點：
[root@master2 ~]# mogadm --trackers=10.201.106.132:7001 host add 10.201.106.133 --ip=10.201.106.133 --status=alive
[root@master2 ~]#
[root@master2 ~]# mogadm --trackers=10.201.106.132:7001 host add 10.201.106.134 --ip=10.201.106.134 --status=alive
[root@master2 ~]# mogadm host list
10.201.106.132 [1]: alive
IP: 10.201.106.132:7500
10.201.106.133 [2]: alive
IP: 10.201.106.133:7500
10.201.106.134 [3]: alive
IP: 10.201.106.134:7500
[root@master2 ~]# 5.3 修改節點狀態
[root@master2 ~]# mogadm host mark 10.201.106.133 down
[root@master2 ~]# mogadm host list
10.201.106.132 [1]: alive
IP: 10.201.106.132:7500
10.201.106.133 [2]: down
IP: 10.201.106.133:7500
10.201.106.134 [3]: alive
IP: 10.201.106.134:7500
[root@master2 ~]# mogadm host mark 10.201.106.133 alive
[root@master2 ~]# mogadm host list
10.201.106.132 [1]: alive
IP: 10.201.106.132:7500
10.201.106.133 [2]: alive
IP: 10.201.106.133:7500
10.201.106.134 [3]: alive
IP: 10.201.106.134:7500
[root@master2 ~]# 6、
6.1 查看所有設備狀態
[root@master2 ~]# mogadm device list
10.201.106.132 [1]: alive
used(G) free(G) total(G) weight(%)
10.201.106.133 [2]: alive
used(G) free(G) total(G) weight(%)
10.201.106.134 [3]: alive
used(G) free(G) total(G) weight(%)6.2 添加設備
[root@master2 ~]# mogadm device add 10.201.106.132 1
[root@master2 ~]# mogadm device list
10.201.106.132 [1]: alive
used(G) free(G) total(G) weight(%)
dev1: alive 0.054 22.864 22.918 100
10.201.106.133 [2]: alive
used(G) free(G) total(G) weight(%)
10.201.106.134 [3]: alive
used(G) free(G) total(G) weight(%)
[root@master2 ~]#
[root@master2 ~]# mogadm device add 10.201.106.133 2
[root@master2 ~]# mogadm device add 10.201.106.134 3
[root@master2 ~]# mogadm device list
10.201.106.132 [1]: alive
used(G) free(G) total(G) weight(%)
dev1: alive 0.055 22.863 22.918 100
10.201.106.133 [2]: alive
used(G) free(G) total(G) weight(%)
dev2: alive 0.054 22.864 22.918 100
10.201.106.134 [3]: alive
used(G) free(G) total(G) weight(%)
dev3: alive 0.457 22.461 22.918 100
[root@master2 ~]# 6.3 查看現在的狀態
[root@master2 ~]# mogadm check
Checking trackers...
127.0.0.1:7001 ... OK
Checking hosts...
[ 1] 10.201.106.132 ... OK
[ 2] 10.201.106.133 ... OK
[ 3] 10.201.106.134 ... OK
Checking devices...
host device size(G) used(G) free(G) use% ob state I/O%
---- ------------ ---------- ---------- ---------- ------ ---------- -----
[ 1] dev1 22.918 0.055 22.864 0.24% writeable 0.1
[ 2] dev2 22.918 0.055 22.864 0.24% writeable 0.0
[ 3] dev3 22.918 0.458 22.460 2.00% writeable 0.0
---- ------------ ---------- ---------- ---------- ------
total: 68.755 0.568 68.188 0.83%
[root@master2 ~]# 7、創建管理域
7.1 創建管理域
[root@master2 ~]# mogadm domain add images
[root@master2 ~]# mogadm domain add file
[root@master2 ~]# mogadm domain list
domain class mindevcount replpolicy hashtype
-------------------- -------------------- ------------- ------------ -------
file default 2 MultipleHosts() NONE
images default 2 MultipleHosts() NONE
[root@master2 ~]# 8 管理域中的類
8.1 查看class
[root@master2 ~]# mogadm class list
domain class mindevcount replpolicy hashtype
-------------------- -------------------- ------------- ------------ -------
file default 2 MultipleHosts() NONE
images default 2 MultipleHosts() NONE
[root@master2 ~]#8.2 在圖片域中添加class
[root@master2 ~]# mogadm class add images jpeg
[root@master2 ~]# mogadm class add images png
[root@master2 ~]# mogadm class add images gif
[root@master2 ~]# mogadm class list
domain class mindevcount replpolicy hashtype
-------------------- -------------------- ------------- ------------ -------
file default 2 MultipleHosts() NONE
images default 2 MultipleHosts() NONE
images gif 2 MultipleHosts() NONE
images jpeg 2 MultipleHosts() NONE
images png 2 MultipleHosts() NONE
[root@master2 ~]# 8.3 class限制，不要副本
[root@master2 ~]# mogadm class add file plaintext --mindevcount=1
[root@master2 ~]# mogadm class list
domain class mindevcount replpolicy hashtype
-------------------- -------------------- ------------- ------------ -------
file default 2 MultipleHosts() NONE
file plaintext 1 MultipleHosts() NONE 8.4 設置class至少3個保留副本
[root@master2 ~]# mogadm class add file html --replpolicy="MultipleHosts(3)"
[root@master2 ~]# mogadm class list
domain class mindevcount replpolicy hashtype
-------------------- -------------------- ------------- ------------ -------
file default 2 MultipleHosts() NONE
file html 2 MultipleHosts(3) NONE 9、文件上傳
9.1 上傳文件
[root@master2 ~]# mogupload --trackers=10.201.106.132:7001 --domain=file --key=‘/fstab.txt‘ --file=‘/etc/fstab‘ --class=plaintext9.1 查看文件信息
[root@master2 ~]# mogfileinfo
Usage: /usr/bin/mogfileinfo --trackers=host --domain=foo --key=‘/hello.jpg‘
[root@master2 ~]# mogfileinfo --trackers=10.201.106.132:7001 --domain=file --key=‘/fstab.txt‘
- file: /fstab.txt
class: plaintext
devcount: 1
domain: file
fid: 3
key: /fstab.txt
length: 1034
- http://10.201.106.132:7500/dev1/0/000/000/0000000003.fid
[root@master2 ~]# mogupload --trackers=10.201.106.132:7001 --domain=images --key=‘/1.jpg‘ --file=‘/root/1.jpg‘ --class=jpeg
[root@master2 ~]# mogfileinfo --trackers=10.201.106.132:7001 --domain=images --key=‘/1.jpg‘
- file: /1.jpg
class: jpeg
devcount: 1
domain: images
fid: 4
key: /1.jpg
length: 8264
- http://10.201.106.132:7500/dev1/0/000/000/0000000004.fid9.2 訪問測試
9.3 上傳圖片
[root@master2 ~]# mogupload --trackers=10.201.106.132:7001 --domain=images --class=jpeg --key=‘/3.jpg‘ --file=‘/root/3.jpg‘
查看圖片信息
[root@master2 ~]# mogfileinfo --trackers=10.201.106.132:7001 --domain=images --key=‘/3.jpg‘
- file: /3.jpg
class: jpeg
devcount: 1
domain: images
fid: 5
key: /3.jpg
length: 18235
- http://10.201.106.134:7500/dev3/0/000/000/0000000005.fid10 顯示keys
10.1 顯示images域下所有keys
[root@master2 ~]# moglistkeys --trackers=10.201.106.132:7001 --domain=images
/1.jpg
/2.jpg
/3.jpg
[root@master2 ~]# 10.2 列出ID
[root@master2 ~]# moglistfids --trackers=10.201.106.132:7001
fid 3
class plaintext
devcount 1
domain file
key /fstab.txt
length 1034
fid 4
class jpeg
devcount 1
domain images
key /1.jpg
length 826411、查看狀態、統計數據
mogstats：mysql客戶端，連接mysql
[root@master2 ~]# mogstats --db_dsn="DBI:mysql:mogilefs:host=10.201.106.132" --db_user="moguser" --db_pass="mogpass" --verbose --stats="devices,files"
Fetching statistics... (devices,files)
... files stats...
... done
... per-device stats...
... done
Statistics for devices...
device host files status
---------- ---------------- ------------ ----------
dev1 10.201.106.132 2 alive
dev3 10.201.106.134 2 alive
---------- ---------------- ------------ ----------
Statistics for files...
domain class files size (m) fullsize (m)
-------------------- ----------- ---------- ----------- -------------
file plaintext 1 0 0
images jpeg 3 0 0
-------------------- ----------- ---------- ----------- -------------
done11.1 顯示所有狀態
[root@master2 ~]# mogstats --db_dsn="DBI:mysql:mogilefs:host=10.201.106.132" --db_user="moguser" --db_pass="mogpass" --verbose --stats="all"12、之前，不能復制問題
12.1 刪除文件
[root@master2 ~]# mogdelete --trackers=10.201.106.132:7001 --domain=images --key=‘/1.jpg‘
[root@master2 ~]# mogdelete --trackers=10.201.106.132:7001 --domain=images --key=‘/2.jpg‘
[root@master2 ~]# mogdelete --trackers=10.201.106.132:7001 --domain=images --key=‘/3.jpg‘
[root@master2 ~]# mogdelete --trackers=10.201.106.132:7001 --domain=file --key=‘/fstab.txt‘12.2 刪除類
[root@master2 ~]# mogadm class delete images jpeg
[root@master2 ~]# mogadm class delete images png
[root@master2 ~]# mogadm class delete images gif
[root@master2 ~]# mogadm class delete file plaintext
[root@master2 ~]# mogadm class delete file html
[root@master2 ~]# mogadm class list
domain class mindevcount replpolicy hashtype
-------------------- -------------------- ------------- ------------ -------
file default 2 MultipleHosts() NONE
images default 2 MultipleHosts() NONE
[root@master2 ~]# 12.3 刪除域
[root@master2 ~]# mogadm domain delete file
[root@master2 ~]# mogadm domain delete images12.4 重新創建
[root@master2 ~]# mogadm domain add imgs
[root@master2 ~]# mogadm domain list
domain class mindevcount replpolicy hashtype
-------------------- -------------------- ------------- ------------ -------
imgs default 2 MultipleHosts() NONE
[root@master2 ~]#
上傳文件測試：
[root@master2 ~]# mogupload --trackers=10.201.106.132:7001 --domain=imgs --key=‘/1.jpg‘ --file=‘/root/1.jpg‘
還是一個：
[root@master2 ~]# mogfileinfo --tracker=10.201.106.132:7001 --domain=imgs --key=‘/1.jpg‘
- file: /1.jpg
class: default
devcount: 1
domain: imgs
fid: 7
key: /1.jpg
length: 8264
- http://10.201.106.133:7500/dev2/0/000/000/0000000007.fid添加新類：
[root@master2 ~]# mogadm class add imgs jpeg --mindevcount=2 --replpolicy="MultipleHosts(2)"
[root@master2 ~]# mogupload --trackers=10.201.106.132:7001 --domain=imgs --class=jpeg --key=‘/2.jpg‘ --file=‘/root/2.jpg‘
Mogilefs雜記（1）