關於 Kerberos 的安裝和 HDFS 配置 kerberos 認證，請參考 HDFS配置kerberos認證。

請先完成 HDFS 配置 Kerberos 認證，再來配置 YARN 整合 Kerberos 認證 ！

參考 使用yum安裝CDH Hadoop叢集 安裝 hadoop 叢集，叢集包括三個節點，每個節點的ip、主機名和部署的元件分配如下：

192.168.56.121        cdh1     NameNode、Hive、ResourceManager、HBase、Kerberos Server
192.168.56.122        cdh2     DataNode、SSNameNode、NodeManager、HBase
192.168.56.123        cdh3     DataNode、HBase、NodeManager
 

注意：hostname 請使用小寫，要不然在整合 kerberos 時會出現一些錯誤。

1. 生成 keytab

在 cdh1 節點，即 KDC server 節點上執行下面命令：

cd /var/kerberos/krb5kdc/

kadmin.local -q "addprinc -randkey yarn/[email protected] "
kadmin.local -q "addprinc -randkey yarn/[email protected] "
kadmin.local -q "addprinc -randkey yarn/[email protected] "

kadmin.local -q "addprinc -randkey mapred/[email protected] "
 

kadmin.local -q "addprinc -randkey mapred/[email protected] "
kadmin.local -q "addprinc -randkey mapred/[email protected] "

kadmin.local -q "xst  -k yarn.keytab  yarn/[email protected] "
kadmin.local -q "xst  -k yarn.keytab  yarn/[email protected] "
kadmin.local -q "xst  -k yarn.keytab  yarn/[email protected] "
 


kadmin.local -q "xst  -k mapred.keytab  mapred/[email protected] "
kadmin.local -q "xst  -k mapred.keytab  mapred/[email protected] "
kadmin.local -q "xst  -k mapred.keytab  mapred/[email protected] "

拷貝 yarn.keytab 和 mapred.keytab 檔案到其他節點的 /etc/hadoop/conf 目錄

$ scp yarn.keytab mapred.keytab cdh1:/etc/hadoop/conf
$ scp yarn.keytab mapred.keytab cdh2:/etc/hadoop/conf
$ scp yarn.keytab mapred.keytab cdh3:/etc/hadoop/conf

並設定許可權，分別在 cdh1、cdh2、cdh3 上執行：

$ ssh cdh1 "cd /etc/hadoop/conf/;chown yarn:hadoop yarn.keytab;chown mapred:hadoop mapred.keytab ;chmod 400 *.keytab"
$ ssh cdh2 "cd /etc/hadoop/conf/;chown yarn:hadoop yarn.keytab;chown mapred:hadoop mapred.keytab ;chmod 400 *.keytab"
$ ssh cdh3 "cd /etc/hadoop/conf/;chown yarn:hadoop yarn.keytab;chown mapred:hadoop mapred.keytab ;chmod 400 *.keytab"

由於 keytab 相當於有了永久憑證，不需要提供密碼(如果修改 kdc 中的 principal 的密碼，則該 keytab 就會失效)，所以其他使用者如果對該檔案有讀許可權，就可以冒充 keytab 中指定的使用者身份訪問 hadoop，所以 keytab 檔案需要確保只對 owner 有讀許可權(0400)

2. 修改 YARN 配置檔案

修改 yarn-site.xml，新增下面配置：

<property>
  <name>yarn.resourcemanager.keytab</name>
  <value>/etc/hadoop/conf/yarn.keytab</value>
</property>
<property>
  <name>yarn.resourcemanager.principal</name> 
  <value>yarn/[email protected]</value>
</property>

<property>
  <name>yarn.nodemanager.keytab</name>
  <value>/etc/hadoop/conf/yarn.keytab</value>
</property>
<property>
  <name>yarn.nodemanager.principal</name> 
  <value>yarn/[email protected]</value>
</property> 
<property>
  <name>yarn.nodemanager.container-executor.class</name>  
  <value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value>
</property> 
<property>
  <name>yarn.nodemanager.linux-container-executor.group</name>
  <value>yarn</value>
</property>

如果想要 YARN 開啟 SSL，則新增：

<property>
  <name>yarn.http.policy</name>
  <value>HTTPS_ONLY</value>
</property>

修改 mapred-site.xml，新增如下配置：

<property>
  <name>mapreduce.jobhistory.keytab</name>
  <value>/etc/hadoop/conf/mapred.keytab</value>
</property> 
<property>
  <name>mapreduce.jobhistory.principal</name> 
  <value>mapred/[email protected]</value>
</property>

如果想要 mapreduce jobhistory 開啟 SSL，則新增：

<property>
  <name>mapreduce.jobhistory.http.policy</name>
  <value>HTTPS_ONLY</value>
</property>

在 /etc/hadoop/conf 目錄下建立 container-executor.cfg 檔案，內容如下：

#configured value of yarn.nodemanager.linux-container-executor.group
yarn.nodemanager.linux-container-executor.group=yarn
#comma separated list of users who can not run applications
banned.users=bin
#Prevent other super-users
min.user.id=0
#comma separated list of system users who CAN run applications
allowed.system.users=root,nobody,impala,hive,hdfs,yarn

設定該檔案許可權：

$ chown root:yarn container-executor.cfg
$ chmod 400 container-executor.cfg

$ ll container-executor.cfg
-r-------- 1 root yarn 354 11-05 14:14 container-executor.cfg

注意：

• container-executor.cfg 檔案讀寫許可權需設定為 400，所有者為 root:yarn。
• yarn.nodemanager.linux-container-executor.group 要同時配置在 yarn-site.xml 和 container-executor.cfg，且其值需要為執行 NodeManager 的使用者所在的組，這裡為 yarn。
• banned.users 不能為空，預設值為 hfds,yarn,mapred,bin
• min.user.id 預設值為 1000，在有些 centos 系統中，使用者最小 id 為500，則需要修改該值
• 確保 yarn.nodemanager.local-dirs 和 yarn.nodemanager.log-dirs 對應的目錄許可權為 755 。

設定 /usr/lib/hadoop-yarn/bin/container-executor 讀寫許可權為 6050 如下：

$ chown root:yarn /usr/lib/hadoop-yarn/bin/container-executor
$ chmod 6050 /usr/lib/hadoop-yarn/bin/container-executor

$ ll /usr/lib/hadoop-yarn/bin/container-executor
---Sr-s--- 1 root yarn 333 11-04 19:11 container-executor

測試是否配置正確：

$ /usr/lib/hadoop-yarn/bin/container-executor --checksetup

記住將修改的上面檔案同步到其他節點：cdh2、cdh3，並再次一一檢查許可權是否正確。

$ cd /etc/hadoop/conf/

$ scp yarn-site.xml mapred-site.xml container-executor.cfg  cdh2:/etc/hadoop/conf/
$ scp yarn-site.xml mapred-site.xml container-executor.cfg  cdh3:/etc/hadoop/conf/

$ ssh cdh2 "cd /etc/hadoop/conf/; chown root:yarn container-executor.cfg ; chmod 400 container-executor.cfg"
$ ssh cdh3 "cd /etc/hadoop/conf/; chown root:yarn container-executor.cfg ; chmod 400 container-executor.cfg"

3. 啟動服務

啟動 ResourceManager

resourcemanager 是通過 yarn 使用者啟動的，故在 cdh1 上先獲取 yarn 使用者的 ticket 再啟動服務：

$ kinit -k -t /etc/hadoop/conf/yarn.keytab yarn/[email protected]
$ service hadoop-yarn-resourcemanager start

然後檢視日誌，確認是否啟動成功。

啟動 NodeManager

resourcemanager 是通過 yarn 使用者啟動的，故在 cdh2 和 cdh3 上先獲取 yarn 使用者的 ticket 再啟動服務：

$ ssh cdh2 "kinit -k -t /etc/hadoop/conf/yarn.keytab yarn/[email protected] ;service hadoop-yarn-nodemanager start"
$ ssh cdh3 "kinit -k -t /etc/hadoop/conf/yarn.keytab yarn/[email protected] ;service hadoop-yarn-nodemanager start"

啟動 MapReduce Job History Server

resourcemanager 是通過 mapred 使用者啟動的，故在 cdh1 上先獲取 mapred 使用者的 ticket 再啟動服務：

$ kinit -k -t /etc/hadoop/conf/mapred.keytab mapred/[email protected]
$ service hadoop-mapreduce-historyserver start

4. 測試

執行一個 mapreduce 的例子：

$ klist
  Ticket cache: FILE:/tmp/krb5cc_1002
  Default principal: yarn/[email protected]

  Valid starting     Expires            Service principal
  11/10/14 11:18:55  11/11/14 11:18:55  krbtgt/[email protected]
    renew until 11/17/14 11:18:55


  Kerberos 4 ticket cache: /tmp/tkt1002
  klist: You have no tickets cached

$ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 10000

如果沒有報錯，則說明配置成功。最後執行的結果為：

Job Finished in 54.56 seconds
Estimated value of Pi is 3.14120000000000000000

如果出現下面錯誤，請檢查環境變數中 HADOOP_YARN_HOME 是否設定正確，並和 yarn.application.classpath 中的保持一致。

14/11/13 11:41:02 INFO mapreduce.Job: Job job_1415849491982_0003 failed with state FAILED due to: Application application_1415849491982_0003 failed 2 times due to AM Container for appattempt_1415849491982_0003_000002 exited with  exitCode: 1 due to: Exception from container-launch.
Container id: container_1415849491982_0003_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
  at org.apache.hadoop.util.Shell.run(Shell.java:455)
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
  at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:281)
  at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
  at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)

Shell output: main : command provided 1
main : user is yarn
main : requested yarn user is yarn


Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
14/11/13 11:41:02 INFO mapreduce.Job: Counters: 0
Job Finished in 13.428 seconds
java.io.FileNotFoundException: File does not exist: hdfs://cdh1:8020/user/yarn/QuasiMonteCarlo_1415850045475_708291630/out/reduce-out