1. 程式人生 > >突破單機多實例Elasticsearch

突破單機多實例Elasticsearch

tps started start 關閉 str var 單機多實例 p s lte

    默認大家都是單機單實例es,在實驗環境下想盡可能模擬各種場景。單機多實例就出來了。。。

實驗拓撲圖

技術分享

01、es安裝這裏就不說了

詳情:http://www.cnblogs.com/xiaochina/p/6852677.html

02、講elasticsearch.yml配置

要做到單機上開多個實例,需要修改ES的默認配置:

node.max_local_storage_nodes: 2 #單機開啟2個實例

配置限制了單節點上可以開啟的ES存儲實例的個數

http.port: 9200

配置是elasticsearch對外提供服務的http端口配置,默認情況下ES會取用9200~9299之間的端口,如果9200被占用就會自動使用9201

在單機多實例的配置中這個配置實際是不需要修改的,但是為了更好地進行配置管理,建議根據需求修改9201 9202

transport.tcp.port: 9300

配置指定了elasticsearch集群內數據通訊使用的端口,默認情況下為9300,與上面的http.port配置類似,ES也會自動為已占用的端口選擇下一個端口號。我們可以將第一個實例的tcp傳輸端口配置為9300,第二實例配置為9301。

discovery.zen.ping.unicast.hosts 由於到了2.x版本之後,ES取消了默認的廣播模式來發現master節點,需要使用該配置來指定發現master節點。這個配置在單機雙實例的配置中需要特別註意下,因為習慣上我們配置時並未指定master節點的tcp端口,如果實例的transport.tcp.port配置為9301,那麽實例啟動後會認為discovery.zen.ping.unicast.hosts中指定的主機tcp端口也是9301,可能導致這些節點無法找到master節點。

該配置中需要指定master節點提供服務的tcp端口。

配置示例:

discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]
  • node.name

    同一主機上的兩個實例需要使用不同的node.name

  • path.data

    同一主機上兩個實例需要對應不同的數據目錄

path.logs

由於默認情況下日誌用集群名稱來命名,因此同一主機兩個實例對應的日誌目錄需要分開

#禁止HTTP
http.enabled: false

配置樣例

node1-es-data

[elk@25 config]$ cat elasticsearch.yml
# 
======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: mvpbang # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-1 node.master: false node.data: true #數據存儲 node.max_local_storage_nodes: 2 #本機最大兩個es transport.tcp.port: 9301 #節點通信端口 http.enabled: false #關閉http接口 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: false bootstrap.system_call_filter: false # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 172.24.0.25 # # Set a custom port for HTTP: # #http.port: 9201 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"] #候選主節點地址 # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligibl e nodes / 2 + 1):# discovery.zen.minimum_master_nodes: 1 # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true

node2-es-data

[elk@25 config]$ cat elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: mvpbang
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-2
node.master: false
node.data: true
node.max_local_storage_nodes: 2
transport.tcp.port: 9302
http.enabled: false
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 172.24.0.25
#
# Set a custom port for HTTP:
#
#http.port: 9202
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligibl
e nodes / 2 + 1):#
discovery.zen.minimum_master_nodes: 1
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

node3-es-master

[elk@25 config]$ cat elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: mvpbang
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-3
node.master: true
node.data: false
node.max_local_storage_nodes: 2
transport.tcp.port: 9301
http.enabled: true
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 172.24.0.26
#
# Set a custom port for HTTP:
#
http.port: 9201
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligibl
e nodes / 2 + 1):#
discovery.zen.minimum_master_nodes: 1
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

node4-es-data

[elk@25 config]$ cat elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: mvpbang
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-4
node.master: false
node.data: false
node.ingest: false
node.max_local_storage_nodes: 2 transport.tcp.port: 9302 http.enabled: true # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: false bootstrap.system_call_filter: false # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 172.24.0.26 # # Set a custom port for HTTP: # http.port: 9202 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"] # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligibl e nodes / 2 + 1):# discovery.zen.minimum_master_nodes: 1 # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true

效果圖

技術分享

註意:復制的elasticsearch文件夾下包含了data文件中示例一的節點數據,需要把示例二data文件下的文件清空

Kibana的elasticsearch.url:配置load-balancedi地址 http://172.24.0.26:9202

學習參考:http://www.tuicool.com/articles/VBVFzyi

https://www.elastic.co/guide/en/kibana/current/production.html#load-balancing #搜索請求節點負載

突破單機多實例Elasticsearch