基於hadoop2.2的namenode與SecondaryNameNode分開配置在不同的計算機
阿新 • • 發佈：2019-01-09
前段時間由於配置hadoop做實驗，在網上看了許多有關hadoop的配置，但是這些配置多數是將namenode和secondaryNameNode配置在同一臺計算機上，這種配置方法如果是做做實驗的還可以，如果應用到實際中，存在較大風險，如果存放namenode的主機出現問題，整個檔案系統將被破壞，嚴重的情況是所有檔案都丟失。現在來配置hadoop2.2,將namenode和secondaryNameNode配置在不同的機器上，這樣的實用價值更大。
1.假設我們已經配置好了hadoop2.2,這時namenode和secondaryNameNode在同一臺機器上，如cloud001;
2.單獨配置secondaryNameNode檔案，指明secondaryNameNode所在的計算機地址，需要masters檔案（類似slaves檔案指明從節點），而我下載的hadoop2.2配置檔案中不包含masters,所以我們新建一個masters檔案，編輯內容為：
sudo vim masterscloud002此處將cloud002作為secondaryNameNode的主機。
3.修改hdfs-site.xml的內容
<!--
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>cloud001:9001</value>
</property>
-->
註釋掉之前使namenode與SecondaryNameNode在同一臺主機的程式碼，並新添如下程式碼
<property> <name>dfs.http.address</name> <value>cloud001:50070</value> <description> The address and the base port where the dfs namenode web ui will listen on. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.namenode.secondary.http-address</name> <value>cloud002:50090</value> </property>
網上也有說要修改core-site.xml的程式碼，而我之前配置過，所以現在就不需要再改了。修改的內容如下：
4.以上修改只是在namenode上修改，在叢集中的其他節點也要作相應修改，簡便方法便是將需要修改的檔案覆蓋掉。<property> <name>fs.checkpoint.period</name> <value>60</value> <description>The number of seconds between two periodic checkpoints. </description> </property> <property> <name>fs.checkpoint.size</name> <value>67108864</value> </property>
5.啟動hdfs,在hadoop目錄下執行：
sbin/start-dfs.sh[email protected]:~/hadoop-2.2.0$ sbin/start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [cloud001]
cloud001: starting namenode, logging to /home/xuhui/hadoop-2.2.0/logs/hadoop-xuhui-namenode-cloud001.out
cloud004: starting datanode, logging to /home/xuhui/hadoop-2.2.0/logs/hadoop-xuhui-datanode-cloud004.out
cloud002: starting datanode, logging to /home/xuhui/hadoop-2.2.0/logs/hadoop-xuhui-datanode-cloud002.out
Starting secondary namenodes [cloud002]
cloud002: starting secondarynamenode, logging to /home/xuhui/hadoop-2.2.0/logs/hadoop-xuhui-secondarynamenode-cloud002.out
starting yarn daemons
starting resourcemanager, logging to /home/xuhui/hadoop-2.2.0/logs/yarn-xuhui-resourcemanager-cloud001.out
cloud004: starting nodemanager, logging to /home/xuhui/hadoop-2.2.0/logs/yarn-xuhui-nodemanager-cloud004.out
cloud002: starting nodemanager, logging to /home/xuhui/hadoop-2.2.0/logs/yarn-xuhui-nodemanager-cloud002.out
[email protected]:~/hadoop-2.2.0$ ls
6.檢查配置成功與否：
[email protected]:~/hadoop-2.2.0$ jps
3490 ResourceManager
3123 NameNode
4784 Jps
[email protected]:~/hadoop-2.2.0$
[email protected]:~$ jps
7162 SecondaryNameNode
7412 NodeManager
6624 DataNode
7698 Jps
[email protected]:~$
[email protected]:~$ jps
6287 NodeManager
6409 Jps
5836 DataNode
[email protected]:~$ 驗證結果很顯然是配置成功的！
7.關閉hdfs:
[email protected]:~/hadoop-2.2.0$ sbin/stop-dfs.sh
Stopping namenodes on [cloud001]
cloud001: stopping namenode
cloud004: stopping datanode
cloud002: stopping datanode
Stopping secondary namenodes [cloud002]
cloud002: stopping secondarynamenode
[email protected]:~/hadoop-2.2.0$