對Hadoop2.7.2文檔的學習-Yarn部分(4)Yarn Application
Writing YARN Applications
Link:http://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
Application submission client向Yarn ResourceManager提交一個Application,RM、NM、AM處理流程。
首先,創建一個YarnClient對象並start它,然後Client可以設置ApplicationContext。為app準備第一個container來containeApplicationMaster
RM在已經指定的Container中啟動ApplicationMaster。AM與YARN集群通信,處理Application的執行。在app啟動的過程中(app的啟動個AM與RM的通信是異步的),AM的主要工作包括:
(1)與RM通信,協商為之後的Containers分配資源(通過AMRMClientAsync對象,AMRMClientAsync.CallbackHandler指定事件的處理方法);
(2)Container分配之後,與NodeManagers通信,啟動它們所在節點的app的Containers(啟動一個Runnable對象,當為Containers分配資源之後,啟動containers。作為啟動Container的一部分,AM需要指定帶有啟動信息的ContainerLaunchContext
在Application執行的過程中,ApplicationMaster通過NMClientAsync對象與NodeManagers通信。所有Containers的事件由NMClientAsync.CallbackHandler處理。
一個callback handler處理Client的start,stop,status update以及error。
原文:
The general concept is that an application submission client submits an application to the YARN ResourceManager
The YARN ResourceManager will then launch the ApplicationMaster (as specified) on an allocated container. The ApplicationMaster communicates with YARN cluster, and handles application execution. It performs operations in an asynchronous fashion. During application launch time, the main tasks of the ApplicationMaster are: a) communicating with the ResourceManager to negotiate and allocate resources for future containers, and b) after container allocation, communicating YARN *NodeManager*s (NMs) to launch application containers on them. Task a) can be performed asynchronously through an AMRMClientAsync object, with event handling methods specified in a AMRMClientAsync.CallbackHandler type of event handler. The event handler needs to be set to the client explicitly. Task b) can be performed by launching a runnable object that then launches containers when there are containers allocated. As part of launching this container, the AM has to specify the ContainerLaunchContext that has the launch information such as command line specification, environment, etc.
對Hadoop2.7.2文檔的學習-Yarn部分(4)Yarn Application