1. 程式人生 > >windows配置hadoop環境變量

windows配置hadoop環境變量

his writing 8.0 exc not -- www. tracker image

1.首先需要下載 hadoop的tar.gz包,目前最新版本是2.9.0 下載地址:

文件共享在最下面

http://hadoop.apache.org/releases.html

前提是JDK必須配置成功

我的hadoop是在 如圖解壓在E盤下 解壓完之後大概是2G多需要一點時間啊

技術分享圖片

2.修改etc\hadoop下的文件(註意以下配置文件下所有的地址都是從Linux集群中的hadoop jar粘貼復制過來的)

4個配置文件

core-site.xml

技術分享圖片

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"
?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value
>/usr/local/hadoop-2.8.0/tmp</value> //linux集群中的地址 </property> </configuration>

mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
 <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.address</name>
    <value>master:10020</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.address</name>
    <value>master:19888</value>
  </property>

</configuration>

hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
      <name>dfs.namenode.secondary.http-address</name>
      <value>master:50090</value>
    </property>
    <property>
      <name>dfs.replication</name>
      <value>2</value>
    </property>
    <property>
      <name>dfs.namenode.name.dir</name>
      <value>file:/usr/local/hadoop-2.8.0/hdfs/name</value>
    </property>
    <property>
      <name>dfs.datanode.data.dir</name>
      <value>file:/usr/local/hadoop-2.8.0/hdfs/data</value>
    </property>

</configuration>

yarn-site.xml

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>

<!-- Site specific YARN configuration properties -->

<property>
          <name>yarn.nodemanager.aux-services</name>
          <value>mapreduce_shuffle</value>
  </property>
  <property>
           <name>yarn.resourcemanager.address</name>
           <value>master:8032</value>
  </property>
  <property>
          <name>yarn.resourcemanager.scheduler.address</name>
          <value>master:8030</value>
  </property>
<property>  
    <name>yarn.log-aggregation-enable</name>  
    <value>true</value>  
</property> 

 <property>
         <name>yarn.resourcemanager.resource-tracker.address</name>
         <value>master:8031</value>
     </property>
     <property>
         <name>yarn.resourcemanager.admin.address</name>
         <value>master:8033</value>
     </property>
     <property>
         <name>yarn.resourcemanager.webapp.address</name>
         <value>master:8088</value>
     </property>

      
</configuration>

3.配置環境變量

此電腦--屬性--高級系統設置--環境變量

技術分享圖片

技術分享圖片

新建系統變量

技術分享圖片

技術分享圖片

變量名:HADOOP_HOME

變量值:你的hadoop位置 E:\hadoop\hadoop-2.8.0\hadoop-2.8.0

編輯Path系統路徑 %HADOOP_HOME%\bin

技術分享圖片

然後一路確定下來

4.編輯hadoop的文件

如下打開 \etc\hadoop\hadoop-enc.cmd文件,修改JAVA_HOME為你自己的jdk路徑 。註意你的JDK安裝在Program Files目錄下,

名稱用名稱用\PROGRA~1\Java 否則中間的空格可能會識別失敗。

技術分享圖片

技術分享圖片

到此為止配置完畢 如果不行的化可能需要點右鍵運行一遍

5.window+r打開 cmd窗口輸入 hadoop測試

技術分享圖片

配置成功

windows配置hadoop環境變量