eclipse 運行 mapreduce程序報錯 No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
報錯信息
17/07/06 17:00:27 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
17/07/06 17:00:27 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
17/07/06 17:00:27 INFO input.FileInputFormat: Total input paths to process : 1
17/07/06 17:00:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/07/06 17:00:27 WARN snappy.LoadSnappy: Snappy native library not loaded
17/07/06 17:00:27 INFO mapred.JobClient: Running job: job_201707060106_0012
17/07/06 17:00:28 INFO mapred.JobClient: map 0% reduce 0%
17/07/06 17:00:34 INFO mapred.JobClient: Task Id : attempt_201707060106_0012_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mapreduce.MapUtil
問題出在工程下沒有job 的jar包
解決辦法
在conf中添加 conf.set("mapred.jar", "hadooptest.jar"); 其中 hadooptest.jar為導出jar包名稱，mapred.jar不變
將工程打包jar文件，放到工程根目錄下，再次運行問題解決
源碼
package com.mapreduce;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
public class MapReduceMain {
public static void main(String[] args) {
Configuration conf = new Configuration();
conf.set("mapred.job.tracker", "test1:9001");
conf.set("mapred.jar", "hadooptest.jar");
try {
Job job = new Job(conf);
job.setJarByClass(MapReduceMain.class);//設置啟動類
job.setMapperClass(MapUtil.class);//設置map類
job.setReducerClass(ReduceUtil.class);//設置reduce類
job.setOutputKeyClass(Text.class);//設置輸出key類型類
job.setOutputValueClass(IntWritable.class);//設置數據value類型類
// job.setNumReduceTasks(1);//設置reduce任務個數，默認為1
//輸入數據所在的文件目錄
FileInputFormat.addInputPath(job, new Path("hdfs://test1:9000/input/"));
//mapreduce執行後輸出數據目錄
FileOutputFormat.setOutputPath(job, new Path("hdfs://test1:9000/output/"));
System.exit(job.waitForCompletion(true)?0:1);
} catch (Exception e) {
e.printStackTrace();
}
}
}
eclipse 運行 mapreduce程序報錯 No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).