1. 程式人生 > >hadoop叢集啟動和關閉shell指令碼

hadoop叢集啟動和關閉shell指令碼

說明:
1、先啟動zookeeper,再啟動hdfs,再啟動yarn。
2、user為hadoop使用的linux使用者。
3、採用SSH登陸到其他機器執行指令碼的方式,且配置了ssh免金鑰登陸。

start-hadoop.sh

user=hadoop
echo start zookeeper...
for zk in master05 master06 master07
do
ssh $user@$zk "source /etc/profile;/master/env/zookeeper/bin/zkServer.sh start"
done
echo zookeeper started

echo start hdfs...
hdfs=master01
ssh $user
@$hdfs "source /etc/profile;/master/env/hadoop/sbin/start-dfs.sh" echo hdfs started echo start yarn... yarn=master03 rsm=master04 ssh $user@$yarn "source /etc/profile;/master/env/hadoop/sbin/start-yarn.sh" ssh $user@$rsm "source /etc/profile;/master/env/hadoop/sbin/yarn-daemon.sh start resourcemanager" echo yarn started

stop-hadoop.sh

user=hadoop
echo stop zookeeper...
for zk in master05 master06 master07
do
ssh $user@$zk "source /etc/profile;/master/env/zookeeper/bin/zkServer.sh stop"
done
echo zookeeper stoped

echo stop hdfs...
hdfs=master01
ssh $user@$hdfs "source /etc/profile;/master/env/hadoop/sbin/stop-dfs.sh"
echo hdfs stoped

echo stop yarn...
yarn=master03
rsm=master04
ssh $user
@$yarn "source /etc/profile;/master/env/hadoop/sbin/stop-yarn.sh" ssh $user@$rsm "source /etc/profile;/master/env/hadoop/sbin/yarn-daemon.sh stop resourcemanager" echo yarn stoped

補充:
互動式shell和非互動式shell、登入shell和非登入shell區別
在登入shell裡,環境資訊需要讀取/etc/profile和~ /.bash_profile, ~/.bash_login, and ~/.profile按順序最先的一個,並執行其中的命令。除非被 –noprofile選項禁止了; 在非登入shell裡,環境資訊只讀取 /etc/bash.bashrc和~/.bashrc
手工執行是屬於登陸shell,指令碼執行資料非登陸shell,而我的linux環境配置中只對/etc/profile進行了jdk1.7等環境的配置,所以指令碼執行/usr/local/zookeeper/bin/zkServer.sh start 啟動zookeeper失敗了
解決方法(下面3個方法任選1):
1、指令碼程式碼中新增“source /etc/profile;” 改為:ssh crxy$i “source /etc/profile;/usr/local/zookeeper/bin/zkServer.sh start”
2、把profile的配置資訊echo到.bashrc中 echo ‘source /etc/profile’ >> ~/.bashrc
3、在/zookeeper/bin/zkEnv.sh的中開始位置新增 export JAVA_HOME=/usr/local/jdk1.7.0_45(就像hadoop中對hadoop-env.sh的配置一樣)

shell 指令碼登陸其他機器操作
1、配置ssh免金鑰登陸。
2、在shell指令碼中執行:ssh linux使用者名稱@主機名或者IP “要執行的命令”