12月13日hadoop-3.0.0釋出正式版啦，試試最新的

Release Notes：

Minimum required Java version increased from Java 7 to Java 8

Support for erasure coding in HDFS

……
Support for more than 2 NameNodes.

Default ports of multiple services have been changed.

but see the release notes for HDFS-9427 and HADOOP-12811 for a list of port changes.
 

下載：

wget http://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-3.0.0/hadoop-3.0.0.tar.gz

解壓：

tar xf hadoop-3.0.0.tar.gz

（一）修改配置檔案

cd hadoop-3.0.0/etc/hadoop

core-site.xml

<property>
    <name>fs.defaultFS</name>
    <value>hdfs://master:9000</value>
</property>

hdfs-site.xml， 這一步需要先建立對應的目錄：

mkdir -p /data/hd3/namenode /data/hd3/datanode
 

內容如下：

    <property>
       <name>dfs.replication</name>
       <value>2</value>
    </property>
    <property>
       <name>dfs.namenode.name.dir</name>
       <value>/data/hd3/namenode</value>
    </property>
    <property>
       <name>dfs.datanode.data.dir</name>
       <value>/data/hd3/datanode</value>
    </property>
 

yarn-site.xml

<property>
    <name>yarn.resourcemanager.hostname</name>
    <value>master</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.env-whitelist</name>
    <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
</property>
<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
</property>
<property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>49152</value>
</property>
<property>
    <name>yarn.scheduler.maximum-allocation-mb</name>
    <value>49152</value>
</property>

mapred-site.xml

    <property>
       <name>mapreduce.framework.name</name>
       <value>yarn</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>

修改slave，注意，這裡改成了workers檔案

slave1
slave2

hadoop-env.sh中新增：（這裡需要自己下載Java 8）

JAVA_HOME=/home/cloud/jdk1.8.0_144

修改profile檔案

sudo vim /etc/profile

新增

export HADOOP_PREFIX=/home/cloud/hadoop-3.0.0
export HADOOP_HDFS_HOME=/home/cloud/hadoop-3.0.0
export HADOOP_CONF_DIR=/home/cloud/hadoop-3.0.0/etc/hadoop

執行生效

source /etc/profile

OK，至此配置全部完成，下面啟動Hadoop。 （二）啟動hadoop 老規矩，首次啟動需要format namenode。

hdfs namenode -format

啟動HDFS

start-dfs.sh

啟動YARN

start-yarn.sh

如果不報錯的話，應該看到如下資訊： 在Master機器上通過JPS命令檢視：

$ jps
6993 SecondaryNameNode
7715 NodeManager
9524 Jps
7371 ResourceManager
6492 NameNode
6669 DataNode

在Slave機器上jps：

$ jps
21360 DataNode
30233 Jps
21643 NodeManager

（三）Web端檢視
訪問 http://10.0.0.1:8088/

訪問 http://10.0.0.1:9870 ，注意，這裡是9870，不是50070了：

（四）OK，大功告成