Elasticsearch6.3.2啟動過程源碼閱讀記錄

阿新 • • 發佈：2018-12-15

周期 msu resource disco aware puts print bootstra dde

Elasticsearch6.3.2啟動過程源碼閱讀記錄

網上有很多關於es的源碼分析，覺得自己技術深度還不夠，所以這些文章只是看源碼過程中的一個筆記，談不上分析。

整個啟動過程以類名.方法名，按順序依次描述如下：

Elasticsearch.main啟動入口類，註冊JVM關閉鉤子用來清理資源。

Command.mainwithoutErrorHandling在es正式啟動之前，加載一些命令：比如 ./elasticsearch -help命令

starts elasticsearch
Option                Description                                               
------                -----------                                               
-E <KeyValuePair>     Configure a setting                                       
-V, --version         Prints elasticsearch version information and exits        
-d, --daemonize       Starts Elasticsearch in the background                    
-h, --help            show help

EnvironmentAwareCommand.execute加載配置參數

        putSystemPropertyIfSettingIsMissing(settings, "path.data", "es.path.data");
        putSystemPropertyIfSettingIsMissing(settings, "path.home", "es.path.home");
        putSystemPropertyIfSettingIsMissing(settings, "path.logs", "es.path.logs");

InternalSettingsPrepare.prePareEnvironment

解析ElasticSearch.yml中的配置參數

Prepares the settings by gathering all elasticsearch system properties, optionally loading the configuration settings,and then replacing all property placeholders.and then replacing all property placeholders.

技術分享圖片

ElasticSearch.execute執行初始化命令。另外在源碼中還有看到一些有趣的註釋，比如必須設置java.io.tmpdir

，這個參數在 config/jvm.options文件中指定。

// a misconfigured java.io.tmpdir can cause hard-to-diagnose problems later, so reject it immediately
        try {
            env.validateTmpFile();
        } catch (IOException e) {
            throw new UserException(ExitCodes.CONFIG, e.getMessage());
        }

Bootstrap.init正式開始啟動ElasticSearch。This method is invoked by {@link Elasticsearch#main(String[])} to startup elasticsearch。

創建節點啟動時需要的環境變量參數

        final Environment environment = createEnvironment(foreground, pidFile, keystore, initialEnv.settings(), initialEnv.configFile());

checkLucene()檢查匹配的Lucene jar包。

創建節點，在第7點中將詳細分析這個過程。

        node = new Node(environment) {
            @Override
            protected void validateNodeBeforeAcceptingRequests(
                final BootstrapContext context,
                final BoundTransportAddress boundTransportAddress, List<BootstrapCheck> checks) throws NodeValidationException {
                BootstrapChecks.check(context, boundTransportAddress, checks);
            }
        };

技術分享圖片

Node.java 構造方法 Node(final Environment environment, Collection<Class<? extends Plugin>> classpathPlugins)。在這個構建方法裏面，完成了創建一個節點所需的各種信息，這個方法非常重要，下面就例舉出幾個節點創建過程中幾個重要的流程：

設置節點環境變量信息(A component that holds all data paths for a single node.)

nodeEnvironment = new NodeEnvironment(tmpSettings, environment);

構造插件服務(PluginService)，

this.pluginsService = new PluginsService(tmpSettings, environment.configFile(), environment.modulesFile(), environment.pluginsFile(), classpathPlugins);

看這個構造方法的註釋：

    /**
     * Constructs a new PluginService
     * @param settings The settings of the system
     * @param modulesDirectory The directory modules exist in, or null if modules should not be loaded from the filesystem
     * @param pluginsDirectory The directory plugins exist in, or null if plugins should not be loaded from the filesystem
     * @param classpathPlugins Plugins that exist in the classpath which should be loaded
     */
    public PluginsService(Settings settings, Path configPath, Path modulesDirectory, Path pluginsDirectory, Collection<Class<? extends Plugin>> classpathPlugins) {

其實就是加載：elasticsearch-6.3.2/modules 和 elasticsearch-6.3.2/plugins兩個目錄下的內容。
技術分享圖片

創建自定義的線程池，節點執行各種任務用的吧。

final ThreadPool threadPool = new ThreadPool(settings, executorBuilders.toArray(new ExecutorBuilder[0]));

創建NodeClient，Client that executes actions on the local node。
```
client = new NodeClient(settings, threadPool);
```
?

AnalysisModule (An internal registry for tokenizer, token filter, char filter and analyzer)各種分詞器。

AnalysisModule analysisModule = new AnalysisModule(this.environment, pluginsService.filterPlugins(AnalysisPlugin.class));

SettingsModule(A module that binds the provided settings to the {@link Settings} interface) 各種配置參數用到。比如 jvm.options 和 elasticsearch.yml裏面配置的各種參數。
```
final SettingsModule settingsModule = new SettingsModule(this.settings, additionalSettings, additionalSettingsFilter);
```
?

節點是集群的一份子，肯定需要集群相關的服務

final ClusterService clusterService = new ClusterService(settings, settingsModule.getClusterSettings(), threadPool,
               ClusterModule.getClusterStateCustomSuppliers(clusterPlugins));

集群信息相關服務(Interface for a class used to gather information about a cluster at regular intervals) 周期性同步集群狀態。
```
final ClusterInfoService clusterInfoService = newClusterInfoService(settings, clusterService, threadPool, client, listener::onNewInfo);
```
?

創建Module

ModulesBuilder modules = new ModulesBuilder();
// plugin modules must be added here, before others or we can get crazy injection errors...
            for (Module pluginModule : pluginsService.createGuiceModules()) {
                modules.add(pluginModule);
            }

比如：SearchModule(Sets up things that can be done at search time like queries, aggregations, and suggesters)

SearchModule searchModule = new SearchModule(settings, false, pluginsService.filterPlugins(SearchPlugin.class));

還有 ActionModule(Builds and binds the generic action map, all {@link TransportAction}s, and {@link ActionFilters}.)

ActionModule actionModule = new ActionModule(false, settings, clusterModule.getIndexNameExpressionResolver(),
                settingsModule.getIndexScopedSettings(), settingsModule.getClusterSettings(), settingsModule.getSettingsFilter(),
                threadPool, pluginsService.filterPlugins(ActionPlugin.class), client, circuitBreakerService, usageService);
            modules.add(actionModule);

還有 DiscoveryModule(A module for loading classes for node discovery)

final DiscoveryModule discoveryModule = new DiscoveryModule(this.settings, threadPool, transportService, namedWriteableRegistry,
                networkService, clusterService.getMasterService(), clusterService.getClusterApplierService(),
                clusterService.getClusterSettings(), pluginsService.filterPlugins(DiscoveryPlugin.class),
                clusterModule.getAllocationService());

看一下，一共都有哪些module: 技術分享圖片

最終關聯了一大批的Module

modules.add(b -> {
                    b.bind(Node.class).toInstance(this);
                    b.bind(NodeService.class).toInstance(nodeService);
                    b.bind(NamedXContentRegistry.class).toInstance(xContentRegistry);
                    b.bind(PluginsService.class).toInstance(pluginsService);
                    b.bind(Client.class).toInstance(client);
                    b.bind(NodeClient.class).toInstance(client);
                    b.bind(Environment.class).toInstance(this.environment);
                    b.bind(ThreadPool.class).toInstance(threadPool);
                    b.bind(NodeEnvironment.class).toInstance(nodeEnvironment);
                    b.bind(ResourceWatcherService.class).toInstance(resourceWatcherService);
                    b.bind(CircuitBreakerService.class).toInstance(circuitBreakerService);
                    b.bind(BigArrays.class).toInstance(bigArrays);
                    b.bind(ScriptService.class).toInstance(scriptModule.getScriptService());
                    b.bind(AnalysisRegistry.class).toInstance(analysisModule.getAnalysisRegistry());
                    b.bind(IngestService.class).toInstance(ingestService);
                    b.bind(UsageService.class).toInstance(usageService);
                    b.bind(NamedWriteableRegistry.class).toInstance(namedWriteableRegistry);
                    b.bind(MetaDataUpgrader.class).toInstance(metaDataUpgrader);
                    b.bind(MetaStateService.class).toInstance(metaStateService);
                    b.bind(IndicesService.class).toInstance(indicesService);
                    b.bind(SearchService.class).toInstance(searchService);
                    b.bind(SearchTransportService.class).toInstance(searchTransportService);
                    b.bind(SearchPhaseController.class).toInstance(new SearchPhaseController(settings,
                        searchService::createReduceContext));
                    b.bind(Transport.class).toInstance(transport);
                    b.bind(TransportService.class).toInstance(transportService);
                    b.bind(NetworkService.class).toInstance(networkService);
                    b.bind(UpdateHelper.class).toInstance(new UpdateHelper(settings, scriptModule.getScriptService()));
                    b.bind(MetaDataIndexUpgradeService.class).toInstance(metaDataIndexUpgradeService);
                    b.bind(ClusterInfoService.class).toInstance(clusterInfoService);
                    b.bind(GatewayMetaState.class).toInstance(gatewayMetaState);
                    b.bind(Discovery.class).toInstance(discoveryModule.getDiscovery());

總之，Node.java的構造方法裏面實現了創建一個ElasticSearch節點所必須的各種信息，想要了解ElasticSearch節點的內部結構，應該就得多看看這個方法裏面的代碼吧。

ModulesBuilder.createInjector 使用了Guice 依賴註入。

            injector = modules.createInjector();

Node.start，前面創建了節點，現在開始啟動節點。(Start the node. If the node is already started, this method is no-op)

先拿到對象實例，再啟動

        injector.getInstance(MappingUpdatedAction.class).setClient(client);
        injector.getInstance(IndicesService.class).start();
        injector.getInstance(IndicesClusterStateService.class).start();
        injector.getInstance(SnapshotsService.class).start();
        injector.getInstance(SnapshotShardsService.class).start();
        injector.getInstance(RoutingService.class).start();
        injector.getInstance(SearchService.class).start();
        nodeService.getMonitorService().start();
//...
        Discovery discovery = injector.getInstance(Discovery.class);
      clusterService.getMasterService().setClusterStatePublisher(discovery::publish);
        discovery.start(); // start before cluster service so that it can set initial state on ClusterApplierService
        clusterService.start();

裏面的每個方法，都值得花時間去深入研究下。哈哈。。。

總結

總的來看，Elasticsearch啟動過程三大步，第一步：加載各種配置信息，這些配置信息既有自定義的配置信息，也有機器的環境變量信息，它們告訴es，我想要創建一個什麽樣的節點。第二步：創建節點，節點具有各種各樣的功能，比如說執行搜索查詢請求、選主、與其他節點同步集群狀態信息……這些功能需要各種服務/插件/模塊Module來實現。第三步：啟動節點，其實就是各種模塊、插件、服務的啟動。

最後放一張整理上上面的9個方法的調用關系圖：
技術分享圖片

原文：https://www.cnblogs.com/hapjin/p/10124269.html

Elasticsearch6.3.2啟動過程源碼閱讀記錄

周期 msu resource disco aware puts print bootstra dde Elasticsearch6.3.2啟動過程源碼閱讀記錄網上有很多關於es的源碼分析，覺得自己技術深度還不夠，所以這些文章只是看源碼過程中的一個筆記，談不上分析。整個

Elasticsearch6.3.2啟動過程源碼閱讀記錄