部署前提
1、三臺主機安裝JDK1.7
2、關閉防火牆、selinux
3、配置靜態IP

一、所用軟體
hadoop-2.8.1
zookeeper-3.4.6

二、機器規劃(對應關係寫入到/etc/hosts)

三、建立目錄並上傳Hadoop、zookeeper
/opt/目錄下建立soft資料夾(三臺)

mkdir
 
   /opt/soft

上傳hadoop-2.8.1、zookeeper-3.4.6 至hadoop000機器/opt/soft目錄，解壓

 tar -zxvf zookeeper-3.4.6.tar.gz
 tar -zxvf hadoop-2.8.1.tar.gz

重新命名

 mv hadoop-2.8.1 hadoop
 mv zookeeper-3.4.6 zookeeper

檢視

[[email protected] soft]# ls
hadoop  hadoop-2.8.1.tar.gz  zookeeper  zookeeper-3.4.6.tar.gz

四、三臺主機SSH互相信任關係配置

五、部署zookeeper
1、設定zookeeper環境變數

vim /etc/profile
export ZOOKEEPER_HOME=/opt/soft/zookeeper
export PATH=$ZOOKEEPER_HOME/bin:$PATH
生效環境變數
source /etc/profile
驗證
[root@hadoop000 zookeeper]# echo $ZOOKEEPER_HOME
/opt/soft/zookeeper

2、進入/opt/soft/zookeeper/conf目錄
將目錄下預設檔案zoo_sample.cfg拷貝一份並命令成zoo.cfg

[root@hadoop000 conf]# cp zoo_sample.cfg zoo.cfg

3、編輯 zoo.cfg檔案新增以下內容

dataDir=/opt/soft/zookeeper/data
server.1=hadoop000:2888:3888
server.2=hadoop001:2888:3888
server.3=hadoop002:2888:3888

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.

dataDir=/opt/soft/zookeeper/data

# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1


server.1=hadoop000:2888:3888
server.2=hadoop001:2888:3888
server.3=hadoop002:2888:3888

4、進入zookeeper目錄

[root@hadoop000 zookeeper]# mkdir data
[root@hadoop000 zookeeper]# touch data/myid 
[root@hadoop000 zookeeper]# echo 1 > data/myid 

5、將hadoop000的zookeeper傳到hadoop001/002/opt/soft/目錄

[root@hadoop000 soft]# scp -r zookeeper 192.168.95.20:/opt/soft/ 
[root@hadoop000 soft]# scp -r zookeeper 192.168.95.30:/opt/soft/  

hadoop001/002做如下操作
[root@hadoop001 zookeeper]# echo 2 > data/myid
[root@hadoop002 zookeeper]# echo 3 > data/myid 

六、部署hadoop
1、部署hadoop環境變數，檢視hadoop目錄

[root@hadoop000 hadoop]# pwd
/opt/soft/hadoop

編輯/etc/profile,新增以下內容

export HADOOP_HOME=/opt/soft/hadoop
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

驗證

[root@hadoop000 hadoop]# echo $HADOOP_HOME
/opt/soft/hadoop

2、編輯hadoop-env.sh
進入/opt/soft/hadoop/etc/hadoop目錄，在hadoop-env.sh新增以下內容（25行開始）

25 export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
26 export  HADOOP_OPTS="$HADOOP_OPTS  -Djava.library.path=$HADOOP_HOME/lib:$HADOOP_HOME/bin/native"

3、修改core-site.xml、hdfs-site.xml、mapred-site.xml、yarn-site.xml四個檔案

將/opt/soft/hadoop/etc/hadoop目錄下原有四個檔案刪除，將配置好的四個檔案放到這個目錄即可
注意：檔案有些配置內容以自己機器配置為準！

4、建立臨時資料夾和分發資料夾

[root@hadoop000 hadoop]# mkdir -p /opt/soft/hadoop/tmp
[root@hadoop000 hadoop]# chmod -R 777 /opt/soft/hadoop/tmp
[root@hadoop000 hadoop]# chown -R root:root /opt/soft/hadoop/tmp

[root@hadoop000 hadoop]# scp -r hadoop root@hadoop001:/opt/soft
[root@hadoop000 hadoop]# scp -r hadoop root@hadoop002:/opt/soft

七、啟動叢集
1、啟動zookeeper

[root@hadoop000 zookeeper]# $ZOOKEEPER_HOME/bin/zkServer.sh start
[root@hadoop001 zookeeper]# $ZOOKEEPER_HOME/bin/zkServer.sh start
[root@hadoop002 zookeeper]# $ZOOKEEPER_HOME/bin/zkServer.sh start

2、啟動journalnode程序

[root@hadoop000 ~]# cd /opt/soft/hadoop/sbin
[root@hadoop000 sbin]# hadoop-daemon.sh start journalnode

[root@hadoop001 ~]# cd /opt/soft/hadoop/sbin
[root@hadoop001 sbin]# hadoop-daemon.sh start journalnode

[root@hadoop002 ~]# cd /opt/soft/hadoop/sbin
[root@hadoop002 sbin]# hadoop-daemon.sh start journalnode

3、Namenode格式化
[[email protected] hadoop]# hadoop namenode -format
……………..
……………..
17/09/02 23:16:50 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1577237506-192.168.137.130-1504365410166
17/09/02 23:16:50 INFO common.Storage: Storage directory /opt/soft/hadoop/data/dfs/name has been successfully formatted.
17/09/02 23:16:50 INFO namenode.FSImageFormatProtobuf: Saving image file /opt/soft/hadoop/data/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
17/09/02 23:16:50 INFO namenode.FSImageFormatProtobuf: Image file /opt/soft/hadoop/data/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 306 bytes saved in 0 seconds.
17/09/02 23:16:51 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
17/09/02 23:16:51 INFO util.ExitUtil: Exiting with status 0
17/09/02 23:16:51 INFO namenode.NameNode: SHUTDOWN_MSG:
/**************************************************
SHUTDOWN_MSG: Shutting down NameNode at hadoop001/192.168.95.10

同步hadoop000元資料到hadoop001(Namenode standby)

[root@hadoop000 hadoop]# scp -r data/ root@hadoop001:/opt/soft/hadoop

4、zookeeper初始化
[[email protected] bin]# hdfs zkfc -formatZK
……………..
……………..
17/09/02 23:19:13 INFO ha.ActiveStandbyElector: Session connected.
17/09/02 23:19:13 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/mycluster in ZK.
17/09/02 23:19:13 INFO zookeeper.ZooKeeper: Session: 0x35e42f121f50000 closed
17/09/02 23:19:13 INFO zookeeper.ClientCnxn: EventThread shut down
17/09/02 23:19:13 INFO tools.DFSZKFailoverController: SHUTDOWN_MSG:
/**************************************************
SHUTDOWN_MSG: Shutting down DFSZKFailoverController at hadoop001/192.168.95.10
**************************************************/

5、啟動HDFS

[[email protected] sbin]# start-dfs.sh
Starting namenodes on [hadoop000 hadoop001]
hadoop000: starting namenode, logging to /opt/soft/hadoop/logs/hadoop-root-namenode-hadoop000.out
hadoop001: starting namenode, logging to /opt/soft/hadoop/logs/hadoop-root-namenode-hadoop001.out
hadoop001: starting datanode, logging to /opt/soft/hadoop/logs/hadoop-root-datanode-hadoop001.out
hadoop000: starting datanode, logging to /opt/soft/hadoop/logs/hadoop-root-datanode-hadoop000.out
hadoop002: starting datanode, logging to /opt/soft/hadoop/logs/hadoop-root-datanode-hadoop002.out
Starting journal nodes [hadoop000 hadoop001 hadoop002]
hadoop000: starting journalnode, logging to /opt/soft/hadoop/logs/hadoop-root-journalnode-hadoop000.out
hadoop001: starting journalnode, logging to /opt/soft/hadoop/logs/hadoop-root-journalnode-hadoop001.out
hadoop002: starting journalnode, logging to /opt/soft/hadoop/logs/hadoop-root-journalnode-hadoop002.out
Starting ZK Failover Controllers on NN hosts [hadoop000 hadoop001]
hadoop001: starting zkfc, logging to /opt/soft/hadoop/logs/hadoop-root-zkfc-hadoop001.out
hadoop000: starting zkfc, logging to /opt/soft/hadoop/logs/hadoop-root-zkfc-hadoop000.out

6、啟動YARN

[[email protected] sbin]# start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /opt/software/hadoop/logs/yarn-root-resourcemanager-hadoop000.out
hadoop002: starting nodemanager, logging to /opt/software/hadoop/logs/yarn-root-nodemanager-hadoop002.out
hadoop001: starting nodemanager, logging to /opt/software/hadoop/logs/yarn-root-nodemanager-hadoop001.out
hadoop000: starting nodemanager, logging to /opt/software/hadoop/logs/yarn-root-nodemanager-hadoop000.out

7、備機啟動Resourcemanager

[[email protected] hadoop]# yarn-daemon.sh start resourcemanager

starting resourcemanager, logging to /opt/soft/hadoop/logs/yarn-root-resourcemanager-hadoop002.out

8、檢視程序

[root@hadoop000 ~]# jps
2488 JournalNode
2641 DFSZKFailoverController
2808 ResourceManager
2280 DataNode
5395 Jps
2179 NameNode
2910 NodeManager
2030 QuorumPeerMain

[root@hadoop001 ~]# jps
1927 QuorumPeerMain
2583 ResourceManager
2403 NodeManager
6255 Jps
2262 DFSZKFailoverController
2173 JournalNode
1998 NameNode
2063 DataNode

[root@hadoop002 ~]# jps
2049 NodeManager
2678 Jps
1921 JournalNode
1851 DataNode
1786 QuorumPeerMain

八、驗證HDFS/YARN的HA
hadoop000 (Namenode active)

hadoop001 （Namenode standby）