1. 程式人生 > >Hadoop 2.3.0-cdh5.0.2升級Hadoop 2.5.0-cdh5.3.1

Hadoop 2.3.0-cdh5.0.2升級Hadoop 2.5.0-cdh5.3.1

一、準備升級包 for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/mkdir /home/hadoop/platform";done for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;scp /home/hadoop/platform/hadoop-2.5.0-cdh5.3.3.tar.gz [email protected]$line:/home/hadoop/platform;done for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/tar -zvxf /home/hadoop/platform/hadoop-2.5.0-cdh5.3.3.tar.gz -C /home/hadoop/platform/";done 二、做好檔案備份資訊:  a、備份namenode資訊,即etc/hadoop/hdfs-site.xml下的dfs.namenode.name.dir所指向的資料夾
 cp -r /data/tmp/dfs/name .


 b、備份config檔案:hadoop-env.sh,yarn-env.sh,core-site.xml,mapred-site.xml,hdfs-site.xml,yarn-site.xml
cp -r $HADOOP_CONF_DIR .
c、檔案資訊 hadoop fsck / -files -blocks -locations > dfs-v-old-fsck-1.log hadoop dfs -lsr / > dfs-v-old-lsr-1.log hadoop dfsadmin -report > dfs-v-old-report-1.log 三、將原來的hadoop備份一下,防止升級失敗回滾 for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/cp -r /opt/hadoop /opt/hadoop2.3.0";done 四,刪除原來的軟連線,新建軟連線到新版本的hadoop 注意:刪除軟連線 /opt/hadoop而不是 /opt/hadoop/, for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/rm -rf  /opt/hadoop";done for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/ln -s /data/platform/hadoop-2.5.0-cdh5.3.1 /opt/hadoop";done 五、建立一個FsImage
hdfs dfsadmin -rollingUpgrade prepare hdfs dfsadmin -rollingUpgrade query檢查滾動Image的狀態。等一會接著重新執行這個命令直到“Proceed with rolling upgrade”訊息出現。 六、執行升級命令 a、關閉standby nn1:
hadoop-daemon.sh stop namenode
b、hdfs namenode -rollingUpgrade started c、關閉active nn2:
hadoop-daemon.sh stop namenode
d、hdfs namenode -rollingUpgrade started e、啟動所yostart-all.sh
遇到如下異常:
File system image contains an old layout version -55. An upgrade to version -59 is required. Please restart NameNode with the "-rollingUpgrade started" option if a rolling upgrade is already started;  or restart NameNode with the "-upgrade" option to start a new upgrade. 修改: /data/platform/hadoopupgrade/name/current/VERSION
layoutVersion=-55 改成layoutVersion=-59 nohup hdfs namenode -rollingUpgrade started & datanode啟動不了 最好執行回滾: start-dfs.sh -rollback 參考資料: http://blog.csdn.net/shubingzhuoxue/article/details/50085407
https://www.zybuluo.com/layor/note/161370