1. 程式人生 > >hadoop安裝、使用過程的異常匯總

hadoop安裝、使用過程的異常匯總

ces quest IE sign jdk1 ice UC -s cer

2017年06月06日 17:27:39 閱讀數:711

1、 啟動yarn,resourceManger啟動失敗,日誌報錯:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [jx-vm04:8031] java.net.BindException: Cannot assign requested address; For more details see: http://wiki.apache.org/hadoop/BindException

定位:
1、netstat –ano | grep ‘8031‘  查不到端口占用;
2、可能是yarn-site.xml配置異常,查看配置信息:
<configuration>

<!-- Site specific YARN configuration properties -->
<!-- 1、指定YARN的老大(ResourceManager)的地址 -->
    <property>
                        <name>yarn.resourcemanager.hostname</name>
                        <value>jx-vm04</value>

    </property>
                <!-- 2、reducer獲取數據的方式 -->
    <property>
                        <name>yarn.nodemanager.aux-services</name>
                        <value>mapreduce_shuffle</value>
     </property>


</configuration>
信息配置正常;
3、查看主機配置文件:
$ cat /etc/hosts
127.0.0.1   localhost
#127.0.1.1  jx-vm04
10.10.75.15 jx-vm04

192.168.120.159   mc.zk.thinkjoy.cn
#10.10.75.16 jx-vm01
192.168.3.16 sd-hjy-vm02

本地主機名配置成映射的外網IP(10.10.75.15),這裏要使用內網IP地址才行
改成:  127.0.1.1  jx-vm04
      或者   192.168.3.15  jx-vm04

改完保存,重啟 start-yarn.sh
查看結果:
/hadoop/hadoop-2.4.1/etc/hadoop$ jps
14338 NameNode
25379 NodeManager
25901 ResourceManager
14468 DataNode
26302 Jps
14666 SecondaryNameNode
啟動成功。

2、配置單機版hbase,啟動後報錯:

hadoop@jx-vm04:/usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin$ ./start-hbase.sh 
starting master, logging to /usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/../logs/hbase-hadoop-master-jx-vm04.out
/usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/hbase-daemon.sh: 行 189: /usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/../logs/hbase-hadoop-master-jx-vm04.out: 權限不夠
head: 無法打開"/usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/../logs/hbase-hadoop-master-jx-vm04.out" 讀取數據: 沒有那個文件或目錄
hadoop@jx-vm04:/usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin$ sudo ./start-hbase.sh 
starting master, logging to /usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/../logs/hbase-root-master-jx-vm04.out
Could not start ZK at requested port of 2181.  ZK was started at port: -1.  Aborting as clients (e.g. shell) will not be able to find this ZK quorum.

可以看到在默認賬戶下提示權限不夠,說明要使用管理員賬號或者對應安裝的權限賬號,因為hbase是在hadoop賬戶下安裝的,而zk是在默認的ubuntu賬戶下安裝的,2181端口被ubuntu下的zk占用,而hbase無權訪問。

解決辦法:
將sudo下的zk服務占用進程殺掉,在hadoop賬戶下啟動zk:
hadoop@jx-vm04:/usr/local/wangpei/cloud/zookeeper/zookeeper-3.4.6/bin$ ./zkServer.sh
JMX enabled by default
啟動hbase:
./start-hbase.sh
starting master, logging to /usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/../logs/hbase-root-master-jx-vm04.out

hbase單機啟動成功:

hadoop@jx-vm04:/usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin$ ps -ef | grep hbase
root     11044     1  0 10:56 pts/3    00:00:00 bash /usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/hbase-daemon.sh --config /usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/../conf foreground_start master
root     11058 11044  6 10:56 pts/3    00:00:13 /usr/java/jdk1.6.0_45//bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p -Xmx1000m -XX:+UseConcMarkSweepGC -Dhbase.log.dir=/usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/../logs -Dhbase.log.file=hbase-root-master-jx-vm04.log -Dhbase.home.dir=/usr/local/wangpei/cloud/hbase/hbase-0.98.24-hadoop2/bin/.. -Dhbase.id.str=root -Dhbase.root.logger=INFO,RFA -Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.master.HMaster start
hadoop   11344 10054  0 10:59 pts/3    00:00:00 grep hbase
  • 1
  • 2
  • 3
  • 4
文章標簽: hadoopyarnhbase

hadoop安裝、使用過程的異常匯總