本文采用Vmware12+Centos7+hadoop2.7.3

一.準備工作

1.設定hostname

#hostnamectl set-hostname develop01

# hostnamectl status

2.配置Notepad++修改Linux檔案

點選Notepad++最右側“show NPPFTP”，點選profile setting，設定如下，

點選連線即可完成連線。

修改/etc/hosts

Ping一下檢視配置是否成功：

3.防火牆和SELINUX配置

# firewall-cmd –state

Running

如果要關閉防火牆：

#systemctl stop firewalld.service

關閉SELINUX

# setenforce 0

修改/etc/selinux/config配置

3.幾個重要配置檔案

/etc/profile

/etc/bashrc

/etc/bash_profile

3.安裝java-1.8.0-openjdk-devel.x86_64

我下載的cenos7 JAVA中目錄中只有jre，所以下載一個，不然後面執行不了JPS命令，設定$JAVA_HOME也會出現問題：

#yum installjava-1.8.0-openjdk-devel.x86_64

4.配置SSH免祕鑰登入

設定免密碼登入前要生成自己的公鑰和私鑰。

# ssh-keygen -t rsa

預設生成後存放在/root/.ssh/id_rsa

# cd /root/.ssh/

[root@localhost .ssh]# ls

id_rsa id_rsa.pub  known_hosts

複製祕鑰：

# ssh-copy-id 192.168.220.129

/usr/bin/ssh-copy-id: INFO: Source ofkey(s) to be installed: "/root/.ssh/id_rsa.pub"

The authenticity of host '192.168.220.129(192.168.220.129)' can't be established.

ECDSA key fingerprint isSHA256:XCqvJVXoOI4nSQrOkD38qZtK4YZUw6QRsuRjViRUdWw.

ECDSA key fingerprint isMD5:41:be:79:d5:a7:32:8f:42:9e:2c:14:c3:e3:08:31:9a.

Are you sure you want to continueconnecting (yes/no)? yes

/usr/bin/ssh-copy-id: INFO: attempting tolog in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remainto be installed -- if you are prompted now it is to install the new keys

[email protected]'s password:

Number of key(s) added: 1

此時在/root/.ssh/生成了一個authorized_keys檔案和id_rsa.pub內容相同。

如果還需要免密碼登入其他機器

# ssh-copy-id 其他需要免密登入的主機域名或IP

二.Apache原生單節點版本部署

# mkdir -p /opt/hadoop_singleNode

# tar -zxf hadoop-2.7.3.tar.gz -C/opt/hadoop_singleNode/

# tar -zxf hbase-1.3.0-bin.tar.gz -C/opt/hadoop_singleNode/

1.配置hadoop

使用Notepad++進入/opt/hadoop_singleNode/hadoop-2.7.3/etc/Hadoop

(1)配置hadoop-env.sh

該配置檔案中主要需要配置JAVA_HOME

在Linux中查詢：# echo $JAVA_HOME為空，說明需要在/etc/profile中配置，需要先找到Java安裝路徑：

[root@localhost hadoop]# which java

/usr/bin/java

[root@localhost hadoop]# ls -lrt/usr/bin/java

lrwxrwxrwx. 1 root root 22 Jan 23 04:35/usr/bin/java -> /etc/alternatives/java

[root@localhost hadoop]# ls -lrt/etc/alternatives/java

lrwxrwxrwx. 1 root root 73 Jan 23 04:35/etc/alternatives/java ->/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64/jre/bin/java

配置/etc/profile

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64

export JRE_HOME=$JAVA_HOME/jre 

exportCLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH 

exportPATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

儲存退出後執行命令生效：

[root@localhost hadoop]# source/etc/profile

[root@localhost hadoop]# echo $JAVA_HOME

/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64

配置hadoop-env.sh

exportJAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64/

(2)配置core-site.xml

<configuration>

<property>

   <name>hadoop.tmp.dir</name>

   <value>/opt/hadoop_singleNode/hadoop-2.7.3/tmp</value>

   <description>A base for other temporarydirectories.</description>

 </property>

 <property>

  <!--fs.default.name指定NameNode的IP地址和埠號-->

   <name>fs.default.name</name>

   <value>hdfs://localhost:54310</value>

   <description>The name of the default file system. A URI whose

 scheme and authority determine the FileSystem implementation. The

 uri's scheme determines the config property (fs.SCHEME.impl) naming

  theFileSystem implementation class. The uri's authority is used to

 determine the host, port, etc. for a filesystem.</description>

 </property>

</configuration>

(3) 配置hdfs-site.xml

<configuration>

<property>

<!--block的副本數，預設為3；你可以設定為1 這樣每個block只會存在一份。-->

 <name>dfs.replication</name>

  <value>1</value>

 <description>Default block replication.

  Theactual number of replications can be specified when the file is created.

  Thedefault is used if replication is not specified in create time.

 </description>

</property>

</configuration>

(4)配置mapred-site.xml

將mapred-site.xml.template重新命名為mapred-site.xml

<configuration>

<property>

 <name>mapred.job.tracker</name>

 <value>localhost:54311</value>

 <description>The host and port that the MapReduce job tracker runs

  at.If "local", then jobs are run in-process as a single map

  andreduce task.

 </description>

</property>

</configuration>

2.啟動hadoop

(1)第一次進入需要格式化hadoop

#/opt/hadoop_singleNode/hadoop-2.7.3/bin/hdfs namenode –format

(2)啟動

#/opt/hadoop_singleNode/hadoop-2.7.3/sbin/start-dfs.sh

# jps

48976 NameNode

49284 SecondaryNameNode

49109 DataNode

50012 Jps

(3)在瀏覽器中檢視資訊

http://192.168.220.129:50070/ 

4)建立一個資料夾試試看

# /opt/hadoop_singleNode/hadoop-2.7.3/bin/hadoopfs -mkdir /test

(5)啟動mapreduce計算框架

#/opt/hadoop_singleNode/hadoop-2.7.3/sbin/start-yarn.sh

starting yarn daemons

starting resourcemanager, logging to/opt/hadoop_singleNode/hadoop-2.7.3/logs/yarn-root-resourcemanager-status.out

root@localhost's password:

localhost: starting nodemanager, logging to/opt/hadoop_singleNode/hadoop-2.7.3/logs/yarn-root-nodemanager-status.out

# jps

48976 NameNode

49284 SecondaryNameNode

49109 DataNode

50534 Jps

50217 ResourceManager

50347 NodeManager

(6)啟動所有（YARN、HDFS、MapReduce）

#/opt/hadoop_singleNode/hadoop-2.7.3/sbin/start-all.sh

# jps

48976 NameNode

49284 SecondaryNameNode

49109 DataNode

51127 Jps

50217 ResourceManager

50347 NodeManager

(7)停止所有

#/opt/hadoop_singleNode/hadoop-2.7.3/sbin/stop-all.sh