hadoop學習資料-03-安裝配置續02
執行:vim hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration>
cp mapred-site.xml.template mapred-site.xml
vim mapred-site.xml
<!--制定mapreduce執行到yarn上,提高演算法-->
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
mapred-site.xml
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <!--制定mapreduce執行到yarn上,提高演算法--> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>
配置yarm-site.xml
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop01</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop01</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
配置slaves檔案
執行vim slaves
hadoop01
刪除localhost 改為hadoop01
JAVA_HOME=/usr/local/src/java/jdk1.7.0_51
JAVA_BIN=/usr/local/src/java/jdk1.7.0_51/bin
HADOOP_HOME=/usr/local/src/hadoop/hadoop-2.7.1
PATH=$JAVA_HOME/bin:$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME JAVA_BIN HADOOP_HOME PATH CLASSPATH
source /etc/profile echo $HADOOP_HOME
也可以這樣測試 hadoop fs -ls /
hadoop namenode -format
重啟linux : reboot
再次重新格式化: hadoop namenode -format 輸入yes
警告異常:ARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
解決連線:http://blog.csdn.net/young_kim1/article/details/50324345
log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR
加以上的那句話就是讓日誌列印錯誤的級別,警告就不列印了,掩耳盜鈴的作用。
cd sbin/ 切換目錄
先啟動這個檔案 start-dfs.sh 是hdfs的啟動檔案。 啟動命令: ./start-dfs.sh 再輸入yes 啟動完成後輸入命令jps
NameNode 相等於賬本節點資訊。
DataNode 相等於資料節點資訊。
SecondaryNameNode 相等於nameNode節點的助理。身上有最新資料的一部分。如果用他來還原的時候會丟失一部分資料。
再啟動start-yarn.sh檔案 是mapreduce處理運算用的(交給yarn來給他運算)的檔案
啟動命令:./start-yarn.sh
啟動完成後輸入命令jps
ResourceManager 相等於運算中的老大。
NodeManager 相等於下面節點的老大。
上面的兩部是分開啟動的。你也可以直接用./start-all.sh命令啟動。./stop-all.sh命令關閉
http://192.168.191.135:50070 來檢視啟動的hadoop服務
可以用以下命令進行操作