1. 程式人生 > >mapreduce JVM引數配置的問題

mapreduce JVM引數配置的問題

在mapred-site.xml中新增引數:

<property>
    <name>mapreduce.map.java.opts</name>
    <value>1536</value>
</property>
<property>
    <name>mapreduce.reduce.java.opts</name>
    <value>3072</value>
</property>

在hive中執行簡單查詢:

select count(1) from sample;

報錯:

Error during job, obtaining debugging information...
Examining task ID: task_1491368313881_0001_m_000000 (and more) from job job_1491368313881_0001
Task with the most failures(4): 
-----
Task ID:
  task_1491368313881_0001_m_000000
URL:
  http://ctdn01:8088/taskdetails.jsp?jobid=job_1491368313881_0001&tipid=task_1491368313881_0001_m_000000
-----
Diagnostic Messages for this Task:
Exception from container-launch.
Container id: container_1491368313881_0001_01_000005
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
at org.apache.hadoop.util.Shell.runCommand(Shell.java:578)
at org.apache.hadoop.util.Shell.run(Shell.java:481)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:763)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

修改調整這兩個引數多次,均不能改正這個問題,現在忽略設定這兩個引數。