FastDFS為互聯網量身定制，充分考慮了冗余備份、負載均衡、線性擴容等機制，並註重高可用、高性能等指標，使用FastDFS很容易搭建一套高性能的文件服務器集群提供文件上傳、下載等服務。

簡介

FastDFS服務端有兩個角色：跟蹤器（tracker）和存儲節點（storage）。跟蹤器主要做調度工作，在訪問上起負載均衡的作用。

存儲節點存儲文件，完成文件管理的所有功能：就是這樣的存儲、同步和提供存取接口，FastDFS同時對文件的metadata進行管理。所謂文件的meta data就是文件的相關屬性，以鍵值對（key value）方式表示，如：width=1024，其中的key為width，value為1024。文件metadata是文件屬性列表，可以包含多個鍵值對。

跟蹤器和存儲節點都可以由一臺或多臺服務器構成。跟蹤器和存儲節點中的服務器均可以隨時增加或下線而不會影響線上服務。其中跟蹤器中的所有服務器都是對等的，可以根據服務器的壓力情況隨時增加或減少。

為了支持大容量，存儲節點（服務器）采用了分卷（或分組）的組織方式。存儲系統由一個或多個卷組成，卷與卷之間的文件是相互獨立的，所有卷的文件容量累加就是整個存儲系統中的文件容量。一個卷可以由一臺或多臺存儲服務器組成，一個卷下的存儲服務器中的文件都是相同的，卷中的多臺存儲服務器起到了冗余備份和負載均衡的作用。

在卷中增加服務器時，同步已有的文件由系統自動完成，同步完成後，系統自動將新增服務器切換到線上提供服務。

當存儲空間不足或即將耗盡時，可以動態添加卷。只需要增加一臺或多臺服務器，並將它們配置為一個新的卷，這樣就擴大了存儲系統的容量。

FastDFS中的文件標識分為兩個部分：卷名和文件名，二者缺一不可。

vip 1 192.168.1.205 綁定域名 image.e3mall.com

192.168.1.0/24 模擬外網

172.16.1.0/24 模擬內網

1.下載所需安裝包

https://github.com/happyfish100/libfastcommon/archive/V1.0.36.zip
https://github.com/happyfish100/fastdfs/archive/master.zip
https://github.com/happyfish100/fastdfs-nginx-module/archive/master.zip
http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz
wget https://github.com/happyfish100/libfastcommon/archive/V1.0.36.zip -O ./libfastcommon.zip
wget https://github.com/happyfish100/fastdfs/archive/master.zip -O ./fastdfs.zip
wget https://github.com/happyfish100/fastdfs-nginx-module/archive/master.zip -O ./fastdfs-nginx-module.zip
wget http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz
[root@tracker1 tools]# ll
total 9928
-rw-r--r-- 1 root root 8234674 Oct 27  2013 apache-tomcat-7.0.47.tar.gz
-rw-r--r-- 1 root root   22192 May 18 12:28 fastdfs-nginx-module.zip
-rw-r--r-- 1 root root  425546 May 18 12:25 fastdfs.zip
-rw-r--r-- 1 root root  481342 May 18 12:25 libfastcommon.zip
-rw-r--r-- 1 root root  981687 Apr 11 09:56 nginx-1.12.2.tar.gz
-rw-r--r-- 1 root root   12248 Dec 24  2014 ngx_cache_purge-2.3.tar.gz
[root@tracker1 tools]# 
把tracker1上的安裝包復制到其他機器
scp ./* [email protected]:/application/tools/
scp ./* [email protected]:/application/tools/
scp ./* [email protected]:/application/tools/
scp ./* [email protected]:/application/tools/
scp ./* [email protected]:/application/tools/
scp ./* [email protected]:/application/tools/
scp ./* [email protected]:/application/tools/

2.所有機器安裝所需庫

yum install readline-devel pcre-devel openssl-devel -y

3.安裝libfastcommon(15-20上安裝)

cd /application/tools/
unzip libfastcommon.zip
cd libfastcommon-1.0.36
./make.sh
./make.sh install

4.安裝fastdfs(15-20上安裝)

cd /application/tools/
unzip fastdfs.zip
cd fastdfs-master/
./make.sh
./make.sh install

5.配置tracker1，2

cd /etc/fdfs/
cp tracker.conf.sample tracker.conf
vim tracker.conf
修改內容
base_path=/fastdfs/tracker
store_lookup=0 (負載策略 0為輪詢 這裏修改為0下面進行測試 後面改回來)
mkdir -p /fastdfs/tracker
/etc/init.d/fdfs_trackerd start
ps -ef | grep fdfs

6.配置storage

cd /etc/fdfs/
cp storage.conf.sample storage.conf
group1
vim storage.conf
base_path=/fastdfs/storage
store_path0=/fastdfs/storage
tracker_server=172.16.1.15:22122
tracker_server=172.16.1.16:22122
group2
vim storage.conf
group_name=group2
base_path=/fastdfs/storage
store_path0=/fastdfs/storage
tracker_server=172.16.1.15:22122
tracker_server=172.16.1.16:22122
所有storage
mkdir -p /fastdfs/storage
/etc/init.d/fdfs_storaged start
cd /fastdfs/storage
tailf logs/storaged.log
[root@storage1-group1 storage]# tailf logs/storaged.log
mkdir data path: FF ...
data path: /fastdfs/storage/data, mkdir sub dir done.
[2018-05-18 14:44:10] INFO - file: storage_param_getter.c, line: 191, use_storage_id=0, id_type_in_filename=ip, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=10.00%, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_compress_binlog_min_interval=0, store_slave_file_use_link=0
[2018-05-18 14:44:10] INFO - file: storage_func.c, line: 257, tracker_client_ip: 172.16.1.17, my_server_id_str: 172.16.1.17, g_server_id_in_filename: 285321408
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 172.16.1.16:22122, as a tracker client, my ip is 172.16.1.17
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 1947, tracker server: #0. 172.16.1.15:22122, my_report_status: -1
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 172.16.1.15:22122, as a tracker client, my ip is 172.16.1.17
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 1947, tracker server: #0. 172.16.1.15:22122, my_report_status: -1
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 1263, tracker server 172.16.1.16:22122, set tracker leader: 172.16.1.16:22122
[2018-05-18 14:44:26] INFO - file: storage_sync.c, line: 2733, successfully connect to storage server 172.16.1.18:23000
可以看到，192.168.1.17這臺設備成功與兩個tracker設備連接了，其中選舉了192.168.1.16作為tracker集群的leader。192.168.1.17和192.168.1.18這兩臺虛擬機同屬一個分組(group1)

7.測試tracker的高可用

172.16.1.16作為tracker集群的leader 現在停掉1.16的tracker
/etc/init.d/fdfs_trackerd stop
在storage1上查看日誌
[root@storage1-group1 storage]# tailf logs/storaged.log
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 172.16.1.16:22122, as a tracker client, my ip is 172.16.1.17
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 1947, tracker server: #0. 172.16.1.15:22122, my_report_status: -1
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 172.16.1.15:22122, as a tracker client, my ip is 172.16.1.17
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 1947, tracker server: #0. 172.16.1.15:22122, my_report_status: -1
[2018-05-18 14:44:26] INFO - file: tracker_client_thread.c, line: 1263, tracker server 172.16.1.16:22122, set tracker leader: 172.16.1.16:22122
[2018-05-18 14:44:26] INFO - file: storage_sync.c, line: 2733, successfully connect to storage server 172.16.1.18:23000
[2018-05-18 14:49:56] ERROR - file: tracker_client_thread.c, line: 1148, tracker server 172.16.1.16:22122, recv data fail, errno: 107, error info: Transport endpoint is not connected.
[2018-05-18 14:49:57] ERROR - file: tracker_client_thread.c, line: 277, connect to tracker server 172.16.1.16:22122 fail, errno: 111, error info: Connection refused
[2018-05-18 14:49:57] INFO - file: tracker_client_thread.c, line: 1263, tracker server 172.16.1.15:22122, set tracker leader: 172.16.1.15:22122
[2018-05-18 14:49:57] ERROR - file: connection_pool.c, line: 130, connect to 172.16.1.16:22122 fail, errno: 111, error info: Connection refused
連接tracker2 1.16失敗 選舉1.15為新leader

重新啟動tracker2 
/etc/init.d/fdfs_trackerd start

在storage1上查看日誌
[2018-05-18 14:56:57] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 172.16.1.16:22122, continuous fail count: 14, as a tracker client, my ip is 172.16.1.17
連接tracker2成功 但leader沒有改別
當我們所有的tracker和storage節點都啟動成功之後，我們可以在任意的一個存儲節點上查看存儲集群的信息，命令：/usr/bin/fdfs_monitor /etc/fdfs/storage.conf
server_count=2, server_index=0
tracker server is 172.16.1.15:22122
group count: 2
Group 1:
group name = group1
disk total space = 17944 MB
disk free space = 15091 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0
Storage 1:
id = 172.16.1.17
ip_addr = 172.16.1.17  ACTIVE
http domain = 
version = 5.12
join time = 2018-05-18 14:43:14
up time = 2018-05-18 14:43:14
total storage = 17944 MB
free storage = 15091 MB
。。。。。。。。。。。。。。。省略
Storage 2:
id = 172.16.1.18
ip_addr = 172.16.1.18  ACTIVE
http domain = 
version = 5.12
join time = 2018-05-18 14:43:14
up time = 2018-05-18 14:43:14
total storage = 17944 MB
free storage = 15120 MB
    。。。。。。。。。。。。。。。省略
Group 2:
group name = group2
disk total space = 17944 MB
disk free space = 15120 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0
Storage 1:
id = 172.16.1.19
ip_addr = 172.16.1.19  ACTIVE
http domain = 
version = 5.12
join time = 2018-05-18 14:43:14
up time = 2018-05-18 14:43:14
total storage = 17944 MB
free storage = 15127 MB
Storage 2:
id = 172.16.1.20
ip_addr = 172.16.1.20  ACTIVE
http domain = 
version = 5.12
join time = 2018-05-18 14:43:14
up time = 2018-05-18 14:43:14
total storage = 17944 MB
free storage = 15120 MB
可以看到tracker Server有兩個，當前提供服務的是172.16.1.15，group的數量是2，第一組的IP有172.16.1.17和172.16.1.18，第二組的IP有172.16.1.19和172.16.1.20，與我們規劃的集群完全一致。

8.tracker和storage集群上傳圖片測試

tracker1
cd /etc/fdfs
cp client.conf.sample client.conf
vim client.conf
# the base path to store log files
base_path=/fastdfs/tracker
# tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address
tracker_server=172.16.1.15:22122
tracker_server=172.16.1.16:22122

上傳/root下的1.png
/usr/bin/fdfs_upload_file  /etc/fdfs/client.conf /root/1.png
group2/M00/00/00/wKgBE1r-fICAcd3kAAHY-4ojheI481.png

其中group2表示這張圖片被保存在了哪個組當中，M00代表磁盤目錄，如果電腦只有一個磁盤那就只有M00， 如果有多個磁盤，那就M01、M02...等等。00/00代表磁盤上的兩級目錄，每級目錄下是從00到FF共256個文件夾，兩級就是256*256個。wKgBE1r-fICAcd3kAAHY-4ojheI481.png表示被存儲到storage上的1.png被重命名的名字，這樣做的目的是為了防止圖片名字重復。
我們到兩組group所在的四臺設備的/fastdfs/storage/data/00/00目錄下查看一下是否有我們剛才上傳的圖片，發現172.16.1.1和172.16.1.20兩臺設備上有該圖片，而172.16.1.17和172.16.1.18兩臺設備上沒有該圖片。這是由於172.16.1.17和172.16.1.18兩臺設備屬於group1，而172.16.1.19和172.16.1.20屬於group2，返回的圖片信息中明確說明了存儲在了group2下面，因此可group1下面是沒有該圖片的。

[root@storage3-group2 /]# cd /fastdfs/storage/data/00/00/
[root@storage3-group2 00]# ls
wKgBE1r-fICAcd3kAAHY-4ojheI481.png
[root@storage3-group2 00]# pwd
/fastdfs/storage/data/00/00
[root@storage3-group2 00]# 
[root@storage4-group2 storage]# cd /fastdfs/storage/data/00/00/
[root@storage4-group2 00]# ls
wKgBE1r-fICAcd3kAAHY-4ojheI481.png
[root@storage4-group2 00]# pwd
/fastdfs/storage/data/00/00
[root@storage4-group2 00]# 
我們在搭建集群的時候，配置的策略是輪詢策略，那麽我們現在再上傳一次該圖片，看是否會存儲到group1下面。如下所示，發現這次返回的路徑信息中顯示存儲到了group1下面。
[root@tracker1 fdfs]# /usr/bin/fdfs_upload_file  /etc/fdfs/client.conf /root/1.png
group1/M00/00/00/wKgBEVr-f4eAAkWHAAHY-4ojheI290.png

9.配置Ngnix(所有storage)

cd /application/tools/
unzip fastdfs-nginx-module.zip
useradd www -M -s /sbin/nologin -u 504
tar xf nginx-1.12.2.tar.gz
cd nginx-1.12.2
./configure --add-module=/application/tools/fastdfs-nginx-module-master/src/ --prefix=/application/nginx-1.12.2 --user=www --group=www --with-http_ssl_module --with-http_stub_status_module 
make && make install
ln -s /application/nginx-1.12.2/ /application/nginx
cd /application/tools/fastdfs-nginx-module-master/src/
cp mod_fastdfs.conf /etc/fdfs/

vim /etc/fdfs/mod_fastdfs.conf
# connect timeout in seconds
# default value is 30s
connect_timeout=10
# network recv and send timeout in seconds
# default value is 30s
network_timeout=30
# the base path to store log files
base_path=/tmp
# if load FastDFS parameters from tracker server
# since V1.12
# default value is false
load_fdfs_parameters_from_tracker=true
# storage sync file max delay seconds
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# since V1.12
# default value is 86400 seconds (one day)
storage_sync_file_max_delay = 86400
# if use storage ID instead of IP address
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# default value is false
# since V1.13
use_storage_id = false
# specify storage ids filename, can use relative or absolute path
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# since V1.13
storage_ids_filename = storage_ids.conf
# FastDFS tracker_server can ocur more than once, and tracker_server format is
#  "host:port", host can be hostname or ip address
# valid only when load_fdfs_parameters_from_tracker is true
tracker_server=172.16.1.15:22122
tracker_server=172.16.1.16:22122
# the port of the local storage server
# the default value is 23000
storage_server_port=23000
# the group name of the local storage server
group_name=group1
# if the url / uri including the group name
# set to false when uri like /M00/00/00/xxx
# set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx
# default value is false
url_have_group_name = true
# path(disk or mount point) count, default value is 1
# must same as storage.conf
store_path_count=1
# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
# must same as storage.conf
store_path0=/fastdfs/storage
#store_path1=/home/yuqing/fastdfs1
# standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level=info
# set the log filename, such as /usr/local/apache2/logs/mod_fastdfs.log
# empty for output to stderr (apache and nginx error_log file)
log_filename=
# response mode when the file not exist in the local file system
## proxy: get the content from other storage server, then send to client
## redirect: redirect to the original storage server (HTTP Header is Location)
response_mode=proxy
# the NIC alias prefix, such as eth in Linux, you can see it by ifconfig -a
# multi aliases split by comma. empty value means auto set by OS type
# this paramter used to get all ip address of the local host
# default values is empty
if_alias_prefix=
# use "#include" directive to include HTTP config file
# NOTE: #include is an include directive, do NOT remove the # before include
#include http.conf
# if support flv
# default value is false
tracker_server=172.16.1.15:22122
tracker_server=172.16.1.16:22122
# the port of the local storage server
# the default value is 23000
storage_server_port=23000
# the group name of the local storage server
group_name=group1
# if the url / uri including the group name
# set to false when uri like /M00/00/00/xxx
# set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx
# default value is false
url_have_group_name = true
# path(disk or mount point) count, default value is 1
# must same as storage.conf
store_path_count=1
# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
# must same as storage.conf
store_path0=/fastdfs/storage
#store_path1=/home/yuqing/fastdfs1
# standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level=info
# set the log filename, such as /usr/local/apache2/logs/mod_fastdfs.log
# empty for output to stderr (apache and nginx error_log file)
log_filename=
# response mode when the file not exist in the local file system
## proxy: get the content from other storage server, then send to client
## redirect: redirect to the original storage server (HTTP Header is Location)
response_mode=proxy
# the NIC alias prefix, such as eth in Linux, you can see it by ifconfig -a
# multi aliases split by comma. empty value means auto set by OS type
# this paramter used to get all ip address of the local host
# default values is empty
if_alias_prefix=
# use "#include" directive to include HTTP config file
# NOTE: #include is an include directive, do NOT remove the # before include
#include http.conf
# if support flv
# default value is false
# since v1.15
flv_support = true
# flv file extension name
# default value is flv
# valid only when load_fdfs_parameters_from_tracker is true
tracker_server=172.16.1.15:22122
tracker_server=172.16.1.16:22122
# the port of the local storage server
# the default value is 23000
storage_server_port=23000
# the group name of the local storage server
group_name=group1
# if the url / uri including the group name
# set to false when uri like /M00/00/00/xxx
# set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx
# default value is false
url_have_group_name = true
# path(disk or mount point) count, default value is 1
# must same as storage.conf
store_path_count=1
# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
# must same as storage.conf
store_path0=/fastdfs/storage
#store_path1=/home/yuqing/fastdfs1
# standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level=info
# set the log filename, such as /usr/local/apache2/logs/mod_fastdfs.log
# empty for output to stderr (apache and nginx error_log file)
log_filename=
# response mode when the file not exist in the local file system
## proxy: get the content from other storage server, then send to client
## redirect: redirect to the original storage server (HTTP Header is Location)
response_mode=proxy
# the NIC alias prefix, such as eth in Linux, you can see it by ifconfig -a
# multi aliases split by comma. empty value means auto set by OS type
# this paramter used to get all ip address of the local host
# default values is empty
if_alias_prefix=
# use "#include" directive to include HTTP config file
# NOTE: #include is an include directive, do NOT remove the # before include
#include http.conf
# if support flv
# default value is false
# since v1.15
flv_support = true
# flv file extension name
# default value is flv
# since v1.15
flv_extension = flv
# set the group count
# set to none zero to support multi-group on this storage server
# set to 0  for single group only
# groups settings section as [group1], [group2], ..., [groupN]
# default value is 0
# since v1.14
group_count = 2
# group settings for group #1
# since v1.14
# when support multi-group on this storage server, uncomment following section
[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/fastdfs/storage
#store_path1=/home/yuqing/fastdfs1
# group settings for group #2
# since v1.14
# when support multi-group, uncomment following section as neccessary
[group2]
group_name=group2
storage_server_port=23000
store_path_count=1
store_path0=/fastdfs/storage

scp /etc/fdfs/mod_fastdfs.conf [email protected]:/etc/fdfs/
scp /etc/fdfs/mod_fastdfs.conf [email protected]:/etc/fdfs/
scp /etc/fdfs/mod_fastdfs.conf [email protected]:/etc/fdfs/

修改1.19與1.20的group_name
vim /etc/fdfs/mod_fastdfs.conf
# the group name of the local storage server
group_name=group2

cp /application/tools/fastdfs-master/conf/http.conf /etc/fdfs/
cp /application/tools/fastdfs-master/conf/mime.types /etc/fdfs/
ln -s /fastdfs/storage/data/ /fastdfs/storage/data/M00

cd /application/nginx/conf/
vim nginx.conf
worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       8888;
        server_name  localhost;
        
        location ~/group([0-9])/M00 {
    root   /fastdfs/storage/data;
              ngx_fastdfs_module;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

scp /application/nginx/conf/nginx.conf [email protected]:/application/nginx/conf/
scp /application/nginx/conf/nginx.conf [email protected]:/application/nginx/conf/
scp /application/nginx/conf/nginx.conf [email protected]:/application/nginx/conf/
/application/nginx/sbin/nginx

tracker1上傳/root下的1.png
[root@tracker1 ~]# /usr/bin/fdfs_upload_file  /etc/fdfs/client.conf /root/1.png
group1/M00/00/00/rBABEVsOdc6AatijAAHY-4ojheI494.png

web訪問測試

配置反向代理(tracker1 tracker2)

cd /application/tools/
useradd www -M -s /sbin/nologin -u 504
tar xf nginx-1.12.2.tar.gz
cd nginx-1.12.2
./configure --prefix=/application/nginx-1.12.2 --user=www --group=www --with-http_ssl_module --with-http_stub_status_module
egrep -v "#|^$" /application/nginx/conf/nginx.conf.default > /application/nginx/conf/nginx.conf

vim /application/nginx/conf/nginx.conf
worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    #group1的服務設置  
    upstream fdfs_group1 {  
         server 172.16.1.17:8888 weight=1 max_fails=2 fail_timeout=30s;  
         server 172.16.1.18:8888 weight=1 max_fails=2 fail_timeout=30s;  
    }  
    #group2的服務設置  
    upstream fdfs_group2 {  
         server 172.16.1.19:8888 weight=1 max_fails=2 fail_timeout=30s;  
         server 172.16.1.20:8888 weight=1 max_fails=2 fail_timeout=30s;  
}
    server {
        listen       80;
        server_name  image.e3mall.com;
        #group1的負載均衡配置  
        location /group1/M00 {  
            proxy_next_upstream http_502 http_504 error timeout invalid_header;  
            #對應group1的服務設置  
proxy_pass http://fdfs_group1; 
            proxy_set_header Host $http_host;  
            proxy_set_header Cookie $http_cookie;  
            proxy_set_header X-Real-IP $remote_addr;  
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;  
            proxy_set_header X-Forwarded-Proto $scheme;  
            client_max_body_size 300m;  
        }
        #group2的負載均衡配置
        location /group2/M00 {
            proxy_next_upstream http_502 http_504 error timeout invalid_header;
            #對應group1的服務設置
proxy_pass http://fdfs_group2;
            proxy_set_header Host $http_host;  
            proxy_set_header Cookie $http_cookie;  
            proxy_set_header X-Real-IP $remote_addr;  
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;  
            proxy_set_header X-Forwarded-Proto $scheme;  
            client_max_body_size 300m;     
        }
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

keepalived配置(tracker1 tracker2)

yum -y install keepalived
vim /etc/keepalived/keepalived.conf
--------------------------------------------------------------------------------tracker1------------------------------------------------------------------------------
! Configuration File for keepalived
global_defs {
   router_id nginx1  #兩臺配置不同
}
vrrp_script chk_status {                      #nginx健康檢查
   script "/server/scripts/chkec_status.sh" 
   interval 2                                #每隔2秒中去執行/etc/keepalived/check_status.sh腳本一次
   weight -2 #腳本執行成功後把192.168.156.11這個節點的優先級降低2
}
vrrp_instance VI_1 {
    state BACKUP #角色
    interface eth1                           #監控網絡接口
    virtual_router_id 51                     #兩臺配置必須一樣
    priority 100 #優先級 優先級大的持有vip
    nopreempt                                #非搶占模式 宕機恢復後不會搶占vip
advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_status
}
    virtual_ipaddress {
        192.168.1.205/24    dev eth0
}
---------------------------------------------------------------------------tracker2------------------------------------------------------------------------------
! Configuration File for keepalived
global_defs {
   router_id nginx2  #兩臺配置不同
}
vrrp_script chk_status {                      #nginx健康檢查
   script "/server/scripts/check_status.sh" 
   interval 2                                #每隔2秒中去執行/etc/keepalived/check_status.sh腳本一次
   weight -2 #腳本執行成功後把192.168.156.11這個節點的優先級降低2
}
vrrp_instance VI_1 {
    state BACKUP #角色
    interface eth1                           #監控網絡接口
    virtual_router_id 51                     #兩臺配置必須一樣
    priority 100 #優先級 優先級大的持有vip
    nopreempt                                #非搶占模式 宕機恢復後不會搶占vip
advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_status
}
    virtual_ipaddress {
        192.168.1.205/24    dev eth0
}

chk_status.sh
#!/bin/sh
ip=`ifconfig eth1 | awk -F "[ :]+" 'NR==2 {print $4}'`
name=$HOSTNAME
A=`ps -C nginx --no-header |wc -l`
B=`netstat -lntp | grep fdfs_trackerd | wc -l`
if [ $A -eq 0 ];then  
  killall keepalived 
  sleep 2
  /application/nginx/sbin/nginx 
    if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then  
      echo -e "$name:$ip 的nginx進程中斷 嘗試重啟失敗" | mail -s "nginx宕掉 已經結束keepalived進程 主備已切換" [email protected]
    else
      /etc/init.d/keepalived restart
      echo -e "$name:$ip 的nginx進程中斷 嘗試重啟成功" | mail -s "nginx宕掉 已經恢復 主備已切換 $name:$ip 已恢復 重新作為備機" [email protected]
     fi
fi
if [ $B -eq 0 ];then  
  killall keepalived 
  sleep 2
  /etc/init.d/fdfs_trackerd stop
  /etc/init.d/fdfs_trackerd start
  sleep 5
    if [ `netstat -lntp | grep fdfs_trackerd | wc -l` -eq 0 ];then  
      echo -e "$name:$ip 的fdfs_trackerd進程中斷 嘗試重啟失敗" | mail -s "fdfs_trackerd宕掉 已經結束keepalived進程 主備已切換" [email protected]
    else
      /etc/init.d/keepalived restart
      echo -e "$name:$ip 的fdfs_trackerd進程中斷 嘗試重啟成功" | mail -s "fdfs_trackerd宕掉 已經恢復 主備已切換 $name:$ip 已恢復 重新作為備機" [email protected]
     fi
fi

chmod 755 /server/scripts/check_status.sh
chkconfig keepalived on
/etc/init.d/keepalived start

郵件設置

vim /etc/mail.rc
set [email protected]
set smtp=smtp.163.com
set [email protected]
set smtp-auth-password=a625013463
set smtp-auth=login

七、集群搭建-Fadfs圖片服務器