hadoop環境搭建與測試
阿新 • • 發佈：2017-08-19
sla pre person n! count track 查看 4.2 lin
搭建參看：
http://blog.csdn.net/w13770269691/article/details/16883663/
查看集群狀態：
/* */ bin]#
[[email protected] hadoop2.2]$
查看各節點狀態：
http://192.168.56.101:50070
查看ResourceManager上cluster執行狀態：
http://192.168.56.101:8088
[[email protected] bin]# hdfs dfsadmin -report Configured Capacity: 36729053184 (34.21 GB) Present Capacity: 13322559491 (12.41 GB) DFS Remaining: 13322240000 (12.41 GB) DFS Used: 319491 (312.00 KB) DFS Used%: 0.00% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 ------------------------------------------------- Datanodes available: 2 (2 total, 0 dead) Live datanodes: Name: 192.168.137.103:50010 (slave2) Hostname: slave2 Decommission Status : Normal Configured Capacity: 18364526592 (17.10 GB) DFS Used: 45056 (44 KB) Non DFS Used: 11702558720 (10.90 GB) DFS Remaining: 6661922816 (6.20 GB) DFS Used%: 0.00% DFS Remaining%: 36.28% Last contact: Thu Nov 06 21:26:34 CST 2014 Name: 192.168.137.102:50010 (slave1) Hostname: slave1 Decommission Status : Normal Configured Capacity: 18364526592 (17.10 GB) DFS Used: 274435 (268.00 KB) Non DFS Used: 11703934973 (10.90 GB) DFS Remaining: 6660317184 (6.20 GB) DFS Used%: 0.00% DFS Remaining%: 36.27% Last contact: Thu Nov 06 21:26:31 CST 2014
[[email protected] bin]# hdfs fsck / -files -blocks Status: HEALTHY Total size: 219351 B Total dirs: 11 Total files: 12 Total symlinks: 0 Total blocks (validated): 10 (avg. block size 21935 B) Minimally replicated blocks: 10 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 2 Number of racks: 1 FSCK ended at Thu Nov 06 21:27:34 CST 2014 in 29 milliseconds The filesystem under path '/' is HEALTHY [[email protected]
在環境搭建的過程中，假設出現不論什麽問題。都要去查看日誌 日誌路徑是：/home/hadoop/hadoop2.2/logs
在配置完畢HADOOP_HOME之後，而且使之生效，那麽接下來就進行測試，啟動hadoop 首先在/文件夾下創建input文件
[[email protected]/]# vim input。在該文件裏輸入例如以下內容：I am a very good person! I love you America ! 將之上傳到hdfs上：[[email protected] /]# hadoop fs -put /input /input 在hadoop的bin文件夾下運行：[[email protected] bin]# ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount /input /output 這樣運行完成之後能夠看到：output文件夾下有兩個文件 [[email protected] ~]# hadoop fs -ls /output Found 2 items -rw-r--r-- 1 root supergroup 0 2014-11-06 21:21 /output/_SUCCESS -rw-r--r-- 1 root supergroup 64 2014-11-06 21:21 /output/part-r-00000 接著能夠查看Wordcount的統計結果： [[email protected] bin]# hadoop fs -cat /output/part-r-00000 ! 1 America 1 I 2 a 1 am 1 good 1 love 1 person! 1 very 1 you 1 [[email protected] bin]#
hadoop環境搭建與測試