1. 程式人生 > >Hadoop多次格式化導致datanode無法啟動

Hadoop多次格式化導致datanode無法啟動

首先看一下報錯資訊:

017-12-14 05:07:57,636 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default>
        at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:409)
        at org.apache.hadoop.hdfs.server.datanode.DataStorage
.addStorageLocations(DataStorage.java:388) at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:556) at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1566) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool
(DataNode.java:1527) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:327) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:266) at org.apache.hadoop.hdfs.server.datanode
.BPServiceActor.run(BPServiceActor.java:746) at java.lang.Thread.run(Thread.java:745) 2017-12-14 05:07:58,922 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid 63404450-ed85-4636-8eac-ea75dba1d424) service to hadoop/192.168.137.5:9000. Exiting. java.io.IOException: All specified directories are failed to load. at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:557) at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1566) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1527) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:327) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:266) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:746) at java.lang.Thread.run(Thread.java:745)

這裡面的日誌有一句報錯非常重要

Incompatible clusterIDs in /tmp/hadoop-hadoop/dfs/data: namenode clusterID = CID-c80f243c-4a07-43f3-9eb8-f40d164a4520; datanode clusterID = CID-3e6fcd99-a2fe-42f3-9ccf-bc257a065eb3

這句話告訴我們,namenode的clusterID和datanode的clusterID不同,導致其無法啟動。原因是我們多次格式化namenode導致兩者id不同,無法啟動。
解決方案:
根據日誌,找到存放clusterID的位置i,根據我的日誌位置為/tmp/hadoop-hadoop/dfs/data

hadoop:hadoop:/tmp/hadoop-hadoop/dfs/name:>ll
total 8
drwxrwxr-x 4 hadoop hadoop 4096 Dec 14 05:35 current
-rw-rw-r-- 1 hadoop hadoop   11 Dec 16 05:41 in_use.lock

將current目錄的VERSION中的clusterID複製,覆蓋到cd /tmp/hadoop-hadoop/dfs/data/current/VERSION的clusterID
使兩個的clusterID相同,就可以解決了