elasticsearch系列三：索引詳解（分詞器、文檔管理、路由詳解）

阿新 • • 發佈：2018-06-18

ces com dex 合並 pda ams 最新 case dbi

一、分詞器

技術分享圖片

1. 認識分詞器

1.1 Analyzer 分析器

在ES中一個Analyzer 由下面三種組件組合而成：

character filter ：字符過濾器，對文本進行字符過濾處理，如處理文本中的html標簽字符。處理完後再交給tokenizer進行分詞。一個analyzer中可包含0個或多個字符過濾器，多個按配置順序依次進行處理。
tokenizer：分詞器，對文本進行分詞。一個analyzer必需且只可包含一個tokenizer。
token filter：詞項過濾器，對tokenizer分出的詞進行過濾處理。如轉小寫、停用詞處理、同義詞處理。一個analyzer可包含0個或多個詞項過濾器，按配置順序進行過濾

。

1.2 如何測試分詞器

POST _analyze
{
  "analyzer": "whitespace",
  "text":     "The quick brown fox."
}

POST _analyze
{
  "tokenizer": "standard",
  "filter":  [ "lowercase", "asciifolding" ],
  "text":      "Is this déja vu?"
}

技術分享圖片

position：第幾個詞

offset：詞的偏移位置

2. 內建的character filter

HTML Strip Character Filter
　　html_strip ：

過濾html標簽，解碼HTML entities like &.
Mapping Character Filter
　　mapping ：用指定的字符串替換文本中的某字符串。
Pattern Replace Character Filter
　　pattern_replace ：進行正則表達式替換。

2.1 HTML Strip Character Filter

POST _analyze
{
  "tokenizer":      "keyword", 
  "char_filter":  [ "html_strip" ],
  "text": "< 
p>I&apos;m so <b>happy</b>!</p>"
}

技術分享圖片

在索引中配置：

PUT my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "tokenizer": "keyword",
          "char_filter": ["my_char_filter"]
        }
      },
      "char_filter": {
        "my_char_filter": {
          "type": "html_strip",
          "escaped_tags": ["b"]
        }
      }
    }
  }
}

escaped_tags 用來指定例外的標簽。如果沒有例外標簽需配置，則不需要在此進行客戶化定義，在上面的my_analyzer中直接使用 html_strip

測試：

POST my_index/_analyze
{
  "analyzer": "my_analyzer",
  "text": "<p>I&apos;m so <b>happy</b>!</p>"
}

2.2 Mapping character filter

官網鏈接：https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-mapping-charfilter.html

PUT my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "tokenizer": "keyword",
          "char_filter": [
            "my_char_filter"
          ]
        }
      },
      "char_filter": {
        "my_char_filter": {
          "type": "mapping",
          "mappings": [
            "? => 0",
            "? => 1",
            "? => 2",
            "? => 3",
            "? => 4",
            "? => 5",
            "? => 6",
            "? => 7",
            "? => 8",
            "? => 9"
          ]
        }
      }
    }
  }
}

測試

POST my_index/_analyze
{
  "analyzer": "my_analyzer",
  "text": "My license plate is ?????"
}

技術分享圖片

2.3 Pattern Replace Character Filter

官網鏈接：https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-pattern-replace-charfilter.html

PUT my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "tokenizer": "standard",
          "char_filter": [
            "my_char_filter"
          ]
        }
      },
      "char_filter": {
        "my_char_filter": {
          "type": "pattern_replace",
          "pattern": "(\\d+)-(?=\\d)",
          "replacement": "$1_"
        }
      }
    }
  }
}

測試

POST my_index/_analyze
{
  "analyzer": "my_analyzer",
  "text": "My credit card is 123-456-789"
}

技術分享圖片

3. 內建的Tokenizer

官網鏈接：https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-tokenizers.html

Standard Tokenizer
Letter Tokenizer
Lowercase Tokenizer
Whitespace Tokenizer
UAX URL Email Tokenizer
Classic Tokenizer
Thai Tokenizer
NGram Tokenizer
Edge NGram Tokenizer
Keyword Tokenizer
Pattern Tokenizer
Simple Pattern Tokenizer
Simple Pattern Split Tokenizer
Path Hierarchy Tokenizer

前面集成的中文分詞器Ikanalyzer中提供的tokenizer：ik_smart 、 ik_max_word

測試tokenizer

POST _analyze
{
  "tokenizer":      "standard", 
  "text": "張三說的確實在理"
}

POST _analyze
{
  "tokenizer":      "ik_smart", 
  "text": "張三說的確實在理"
}

4. 內建的Token Filter

ES中內建了很多Token filter ，詳細了解：https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-tokenizers.html

Lowercase Token Filter ：lowercase 轉小寫
Stop Token Filter ：stop 停用詞過濾器
Synonym Token Filter： synonym 同義詞過濾器

說明：中文分詞器Ikanalyzer中自帶有停用詞過濾功能。

4.1 Synonym Token Filter 同義詞過濾器

PUT /test_index
{
    "settings": {
        "index" : {
            "analysis" : {
                "analyzer" : {
                    "my_ik_synonym" : {
                        "tokenizer" : "ik_smart",
                        "filter" : ["synonym"]
                    }
                },
                "filter" : {
                    "synonym" : {
                        "type" : "synonym",
                         <!-- synonyms_path：指定同義詞文件（相對config的位置）-->
                        "synonyms_path" : "analysis/synonym.txt"
                    }
                }
            }
        }
    }
}

同義詞定義格式

ES同義詞格式支持 solr、 WordNet 兩種格式。

在analysis/synonym.txt中用solr格式定義如下同義詞

張三,李四
電飯煲,電飯鍋 => 電飯煲
電腦 => 計算機,computer

註意：

文件一定要UTF-8編碼

一行一類同義詞，=> 表示標準化為

測試：通過例子的結果了解同義詞的處理行為

POST test_index/_analyze
{
  "analyzer": "my_ik_synonym",
  "text": "張三說的確實在理"
}

POST test_index/_analyze
{
  "analyzer": "my_ik_synonym",
  "text": "我想買個電飯鍋和一個電腦"
}

5. 內建的Analyzer

官網鏈接：

https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-analyzers.html

Standard Analyzer
Simple Analyzer
Whitespace Analyzer
Stop Analyzer
Keyword Analyzer
Pattern Analyzer
Language Analyzers
Fingerprint Analyzer

集成的中文分詞器Ikanalyzer中提供的Analyzer：ik_smart 、 ik_max_word

內建的和集成的analyzer可以直接使用。如果它們不能滿足我們的需要，則我們可自己組合字符過濾器、分詞器、詞項過濾器來定義自定義的analyzer

5.1 自定義 Analyzer

配置參數：

技術分享圖片

PUT my_index8
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_ik_analyzer": {
          "type": "custom",
          "tokenizer": "ik_smart",
          "char_filter": [
            "html_strip"
          ],
          "filter": [
             "synonym"
          ]
        }
      },
      "filter": {
        "synonym": {
          "type": "synonym",
          "synonyms_path": "analysis/synonym.txt"
        }
      }    }  }}

5.2 為字段指定分詞器

PUT my_index8/_mapping/_doc
{
  "properties": {
    "title": {
        "type": "text",
        "analyzer": "my_ik_analyzer"
    }
  }
}

如果該字段的查詢需要使用不同的analyzer

PUT my_index8/_mapping/_doc
{
  "properties": {
    "title": {
        "type": "text",
        "analyzer": "my_ik_analyzer",
        "search_analyzer": "other_analyzer" 
    }
  }
}

測試結果

PUT my_index8/_doc/1
{
  "title": "張三說的確實在理"
}

GET /my_index8/_search
{
  "query": {
    "term": {
      "title": "張三"
    }
  }
}

5.3 為索引定義個default分詞器

PUT /my_index10
{
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "tokenizer": "ik_smart",
          "filter": [
            "synonym"
          ]
        }
      },
      "filter": {
        "synonym": {
          "type": "synonym",
          "synonyms_path": "analysis/synonym.txt"
        }
      }
    }
  },
"mappings": {
    "_doc": {
      "properties": {
        "title": {
          "type": "text"
        }
      }
    }
  }
}

測試結果：

PUT my_index10/_doc/1
{
  "title": "張三說的確實在理"
}

GET /my_index10/_search
{
  "query": {
    "term": {
      "title": "張三"
    }
  }
}

6. Analyzer的使用順序

我們可以為每個查詢、每個字段、每個索引指定分詞器。

在索引階段ES將按如下順序來選用分詞：

首先選用字段mapping定義中指定的analyzer
字段定義中沒有指定analyzer，則選用 index settings中定義的名字為default 的analyzer。
如index setting中沒有定義default分詞器，則使用 standard analyzer.

查詢階段ES將按如下順序來選用分詞：

The analyzer defined in a full-text query.
The search_analyzer defined in the field mapping.
The analyzer defined in the field mapping.
An analyzer named default_search in the index settings.
An analyzer named default in the index settings.
The standard analyzer.

二、文檔管理

技術分享圖片

1. 新建文檔

指定文檔id，新增/修改

PUT twitter/_doc/1
{
    "id": 1,
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

新增，自動生成文檔id

POST twitter/_doc/
{
    "id": 1,
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

返回結果說明：

技術分享圖片

2. 獲取單個文檔

HEAD twitter/_doc/11

GET twitter/_doc/1

不獲取文檔的source：

GET twitter/_doc/1?_source=false

獲取文檔的source：

GET twitter/_doc/1/_source

{
  "_index": "twitter",
  "_type": "_doc",
  "_id": "1",
  "_version": 2,
  "found": true,
  "_source": {
    "id": 1,
    "user": "kimchy",
    "post_date": "2009-11-15T14:12:12",
    "message": "trying out Elasticsearch"
  }}

獲取存儲字段

PUT twitter11
{
   "mappings": {
      "_doc": {
         "properties": {
            "counter": {
               "type": "integer",
               "store": false
            },
            "tags": {
               "type": "keyword",
               "store": true
            } }   }  }}

PUT twitter11/_doc/1
{
    "counter" : 1,
    "tags" : ["red"]
}

GET twitter11/_doc/1?stored_fields=tags,counter

技術分享圖片

3. 獲取多個文檔 _mget

方式1：

GET /_mget
{
    "docs" : [
        {
            "_index" : "twitter",
            "_type" : "_doc",
            "_id" : "1"
        },
        {
            "_index" : "twitter",
            "_type" : "_doc",
            "_id" : "2"
            "stored_fields" : ["field3", "field4"]
        }
    ]
}

方式2：

GET /twitter/_mget
{
    "docs" : [
        {
            "_type" : "_doc",
            "_id" : "1"
        },
        {
            "_type" : "_doc",
            "_id" : "2"
        }
    ]
}

方式3：

GET /twitter/_doc/_mget
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "2"
        }
    ]
}

方式4：

GET /twitter/_doc/_mget
{
    "ids" : ["1", "2"]
}

4. 刪除文檔

指定文檔id進行刪除

DELETE twitter/_doc/1

用版本來控制刪除

DELETE twitter/_doc/1?version=1

返回結果：

{
    "_shards" : {
        "total" : 2,
        "failed" : 0,
        "successful" : 2
    },
    "_index" : "twitter",
    "_type" : "_doc",
    "_id" : "1",
    "_version" : 2,
    "_primary_term": 1,
    "_seq_no": 5,
    "result": "deleted"
}

查詢刪除

POST twitter/_delete_by_query
{
  "query": { 
    "match": {
      "message": "some message"
    }
  }
}

當有文檔有版本沖突時，不放棄刪除操作（記錄沖突的文檔，繼續刪除其他復合查詢的文檔）

POST twitter/_doc/_delete_by_query?conflicts=proceed
{
  "query": {
    "match_all": {}
  }
}

通過task api 來查看查詢刪除任務

GET _tasks?detailed=true&actions=*/delete/byquery

查詢具體任務的狀態

GET /_tasks/taskId:1

取消任務

POST _tasks/task_id:1/_cancel

5. 更新文檔

指定文檔id進行修改

PUT twitter/_doc/1
{
    "id": 1,
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

樂觀鎖並發更新控制

PUT twitter/_doc/1?version=1
{
    "id": 1,
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

返回結果

{
  "_index": "twitter",
  "_type": "_doc",
  "_id": "1",
  "_version": 3,
  "result": "updated",
  "_shards": {
    "total": 3,
    "successful": 1,
    "failed": 0
  },
  "_seq_no": 2,
  "_primary_term": 3
}

6.Scripted update 通過腳本來更新文檔

6.1 準備一個文檔

PUT uptest/_doc/1
{
    "counter" : 1,
    "tags" : ["red"]
}

6.2、對文檔1的counter + 4

POST uptest/_doc/1/_update
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    }
}

6.3、往數組中加入元素

POST uptest/_doc/1/_update
{
    "script" : {
        "source": "ctx._source.tags.add(params.tag)",
        "lang": "painless",
        "params" : {
            "tag" : "blue"
        }
    }
}

腳本說明：painless是es內置的一種腳本語言，ctx執行上下文對象（通過它還可訪問_index, _type, _id, _version, _routing and _now (the current timestamp) ），params是參數集合

說明：腳本更新要求索引的_source 字段是啟用的。更新執行流程：

a、獲取到原文檔
b、通過_source字段的原始數據，執行腳本修改。
c、刪除原索引文檔
d、索引修改後的文檔
它只是降低了一些網絡往返，並減少了get和索引之間版本沖突的可能性。

6.4、添加一個字段

POST uptest/_doc/1/_update
{
    "script" : "ctx._source.new_field = ‘value_of_new_field‘"
}

6.5、移除一個字段

POST uptest/_doc/1/_update
{
    "script" : "ctx._source.remove(‘new_field‘)"
}

6.6、判斷刪除或不做什麽

POST uptest/_doc/1/_update
{
    "script" : {
        "source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = ‘delete‘ } else { ctx.op = ‘none‘ }",
        "lang": "painless",
        "params" : {
            "tag" : "green"
        }
    }
}

6.7、合並傳人的文檔字段進行更新

POST uptest/_doc/1/_update
{
    "doc" : {
        "name" : "new_name"
    }
}

6.8、再次執行7，更新內容相同，不需做什麽

{
  "_index": "uptest",
  "_type": "_doc",
  "_id": "1",
  "_version": 4,
  "result": "noop",
  "_shards": {
    "total": 0,
    "successful": 0,
    "failed": 0
  }
}

6.9、設置不做noop檢測

POST uptest/_doc/1/_update
{
    "doc" : {
        "name" : "new_name"
    },
    "detect_noop": false
}

什麽是noop檢測？

即已經執行過的腳本不再執行

6.10、upsert 操作：如果要更新的文檔存在，則執行腳本進行更新，如不存在，則把 upsert中的內容作為一個新文檔寫入。

POST uptest/_doc/1/_update
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    },
    "upsert" : {
        "counter" : 1
    }
}

7. 通過條件查詢來更新文檔

滿足查詢條件的才更新

POST twitter/_update_by_query
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}

8. 批量操作

批量操作API /_bulk 讓我們可以在一次調用中執行多個索引、刪除操作。這可以大大提高索引數據的速度。批量操作內容體需按如下以新行分割的json結構格式給出：

語法：

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

說明：

action_and_meta_data: action可以是 index, create, delete and update ，meta_data 指: _index ,_type,_id 請求端點可以是: /_bulk, /{index}/_bulk, {index}/{type}/_bulk

示例：

POST _bulk
{ "index" : { "_index" : "test", "_type" : "_doc", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_type" : "_doc", "_id" : "2" } }
{ "create" : { "_index" : "test", "_type" : "_doc", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "_doc", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }

8.1 curl + json 文件批量索引多個文檔

curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@accounts.json"

accounts.json：

{"index":{"_id":"1"}}
{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"[email protected]","city":"Brogan","state":"IL"}
{"index":{"_id":"6"}}
{"account_number":6,"balance":5686,"firstname":"Hattie","lastname":"Bond","age":36,"gender":"M","address":"671 Bristol Street","employer":"Netagy","email":"[email protected]","city":"Dante","state":"TN"}
{"index":{"_id":"13"}}
{"account_number":13,"balance":32838,"firstname":"Nanette","lastname":"Bates","age":28,"gender":"F","address":"789 Madison Street","employer":"Quility","email":"[email protected]","city":"Nogal","state":"VA"}
{"index":{"_id":"18"}}
{"account_number":18,"balance":4180,"firstname":"Dale","lastname":"Adams","age":33,"gender":"M","address":"467 Hutchinson Court","employer":"Boink","email":"[email protected]","city":"Orick","state":"MD"}
{"index":{"_id":"20"}}
{"account_number":20,"balance":16418,"firstname":"Elinor","lastname":"Ratliff","age":36,"gender":"M","address":"282 Kings Place","employer":"Scentric","email":"[email protected]","city":"Ribera","state":"WA"}
{"index":{"_id":"25"}}
{"account_number":25,"balance":40540,"firstname":"Virginia","lastname":"Ayala","age":39,"gender":"F","address":"171 Putnam Avenue","employer":"Filodyne","email":"[email protected]","city":"Nicholson","state":"PA"}
{"index":{"_id":"32"}}
{"account_number":32,"balance":48086,"firstname":"Dillard","lastname":"Mcpherson","age":34,"gender":"F","address":"702 Quentin Street","employer":"Quailcom","email":"[email protected]","city":"Veguita","state":"IN"}
{"index":{"_id":"37"}}
{"account_number":37,"balance":18612,"firstname":"Mcgee","lastname":"Mooney","age":39,"gender":"M","address":"826 Fillmore Place","employer":"Reversus","email":"[email protected]","city":"Tooleville","state":"OK"}
{"index":{"_id":"44"}}
{"account_number":44,"balance":34487,"firstname":"Aurelia","lastname":"Harding","age":37,"gender":"M","address":"502 Baycliff Terrace","employer":"Orbalix","email":"[email protected]","city":"Yardville","state":"DE"}
{"index":{"_id":"49"}}
{"account_number":49,"balance":29104,"firstname":"Fulton","lastname":"Holt","age":23,"gender":"F","address":"451 Humboldt Street","employer":"Anocha","email":"[email protected]","city":"Sunriver","state":"RI"}
{"index":{"_id":"51"}}
{"account_number":51,"balance":14097,"firstname":"Burton","lastname":"Meyers","age":31,"gender":"F","address":"334 River Street","employer":"Bezal","email":"[email protected]","city":"Jacksonburg","state":"MO"}
{"index":{"_id":"56"}}
{"account_number":56,"balance":14992,"firstname":"Josie","lastname":"Nelson","age":32,"gender":"M","address":"857 Tabor Court","employer":"Emtrac","email":"[email protected]","city":"Sunnyside","state":"UT"}
{"index":{"_id":"63"}}
{"account_number":63,"balance":6077,"firstname":"Hughes","lastname":"Owens","age":30,"gender":"F","address":"510 Sedgwick Street","employer":"Valpreal","email":"[email protected]","city":"Guilford","state":"KS"}
{"index":{"_id":"68"}}
{"account_number":68,"balance":44214,"firstname":"Hall","lastname":"Key","age":25,"gender":"F","address":"927 Bay Parkway","employer":"Eventex","email":"[email protected]","city":"Shawmut","state":"CA"}
{"index":{"_id":"70"}}
{"account_number":70,"balance":38172,"firstname":"Deidre","lastname":"Thompson","age":33,"gender":"F","address":"685 School Lane","employer":"Netplode","email":"[email protected]","city":"Chestnut","state":"GA"}
{"index":{"_id":"75"}}
{"account_number":75,"balance":40500,"firstname":"Sandoval","lastname":"Kramer","age":22,"gender":"F","address":"166 Irvington Place","employer":"Overfork","email":"[email protected]","city":"Limestone","state":"NH"}
{"index":{"_id":"82"}}
{"account_number":82,"balance":41412,"firstname":"Concetta","lastname":"Barnes","age":39,"gender":"F","address":"195 Bayview Place","employer":"Fitcore","email":"[email protected]","city":"Summerfield","state":"NC"}
{"index":{"_id":"87"}}
{"account_number":87,"balance":1133,"firstname":"Hewitt","lastname":"Kidd","age":22,"gender":"M","address":"446 Halleck Street","employer":"Isologics","email":"[email protected]","city":"Coalmont","state":"ME"}
{"index":{"_id":"94"}}
{"account_number":94,"balance":41060,"firstname":"Brittany","lastname":"Cabrera","age":30,"gender":"F","address":"183 Kathleen Court","employer":"Mixers","email":"[email protected]","city":"Cornucopia","state":"AZ"}
{"index":{"_id":"99"}}
{"account_number":99,"balance":47159,"firstname":"Ratliff","lastname":"Heath","age":39,"gender":"F","address":"806 Rockwell Place","employer":"Zappix","email":"[email protected]","city":"Shaft","state":"ND"}
{"index":{"_id":"102"}}
{"account_number":102,"balance":29712,"firstname":"Dena","lastname":"Olson","age":27,"gender":"F","address":"759 Newkirk Avenue","employer":"Hinway","email":"[email protected]","city":"Choctaw","state":"NJ"}
{"index":{"_id":"107"}}
{"account_number":107,"balance":48844,"firstname":"Randi","lastname":"Rich","age":28,"gender":"M","address":"694 Jefferson Street","employer":"Netplax","email":"[email protected]","city":"Bellfountain","state":"SC"}
{"index":{"_id":"114"}}
{"account_number":114,"balance":43045,"firstname":"Josephine","lastname":"Joseph","age":31,"gender":"F","address":"451 Oriental Court","employer":"Turnabout","email":"[email protected]","city":"Sedley","state":"AL"}
{"index":{"_id":"119"}}
{"account_number":119,"balance":49222,"firstname":"Laverne","lastname":"Johnson","age":28,"gender":"F","address":"302 Howard Place","employer":"Senmei","email":"[email protected]","city":"Herlong","state":"DC"}
{"index":{"_id":"121"}}
{"account_number":121,"balance":19594,"firstname":"Acevedo","lastname":"Dorsey","age":32,"gender":"M","address":"479 Nova Court","employer":"Netropic","email":"[email protected]","city":"Islandia","state":"CT"}
{"index":{"_id":"126"}}
{"account_number":126,"balance":3607,"firstname":"Effie","lastname":"Gates","age":39,"gender":"F","address":"620 National Drive","employer":"Digitalus","email":"[email protected]","city":"Blodgett","state":"MD"}
{"index":{"_id":"133"}}
{"account_number":133,"balance":26135,"firstname":"Deena","lastname":"Richmond","age":36,"gender":"F","address":"646 Underhill Avenue","employer":"Sunclipse","email":"[email protected]","city":"Austinburg","state":"SC"}
{"index":{"_id":"138"}}
{"account_number":138,"balance":9006,"firstname":"Daniel","lastname":"Arnold","age":39,"gender":"F","address":"422 Malbone Street","employer":"Ecstasia","email":"[email protected]","city":"Gardiner","state":"MO"}
{"index":{"_id":"140"}}
{"account_number":140,"balance":26696,"firstname":"Cotton","lastname":"Christensen","age":32,"gender":"M","address":"878 Schermerhorn Street","employer":"Prowaste","email":"[email protected]","city":"Mayfair","state":"LA"}
{"index":{"_id":"145"}}
{"account_number":145,"balance":47406,"firstname":"Rowena","lastname":"Wilkinson","age":32,"gender":"M","address":"891 Elton Street","employer":"Asimiline","email":"[email protected]","city":"Ripley","state":"NH"}
{"index":{"_id":"152"}}
{"account_number":152,"balance":8088,"firstname":"Wolfe","lastname":"Rocha","age":21,"gender":"M","address":"457 Guernsey Street","employer":"Hivedom","email":"[email protected]","city":"Adelino","state":"MS"}
{"index":{"_id":"157"}}
{"account_number":157,"balance":39868,"firstname":"Claudia","lastname":"Terry","age":20,"gender":"F","address":"132 Gunnison Court","employer":"Lumbrex","email":"[email protected]","city":"Castleton","state":"MD"}
{"index":{"_id":"164"}}
{"account_number":164,"balance":9101,"firstname":"Cummings","lastname":"Little","age":26,"gender":"F","address":"308 Schaefer Street","employer":"Comtrak","email":"[email protected]","city":"Chaparrito","state":"WI"}
{"index":{"_id":"169"}}
{"account_number":169,"balance":45953,"firstname":"Hollie","lastname":"Osborn","age":34,"gender":"M","address":"671 Seaview Court","employer":"Musaphics","email":"[email protected]","city":"Hanover","state":"GA"}
{"index":{"_id":"171"}}
{"account_number":171,"balance":7091,"firstname":"Nelda","lastname":"Hopper","age":39,"gender":"M","address":"742 Prospect Place","employer":"Equicom","email":"[email protected]","city":"Finderne","state":"SC"}
{"index":{"_id":"176"}}
{"account_number":176,"balance":18607,"firstname":"Kemp","lastname":"Walters","age":28,"gender":"F","address":"906 Howard Avenue","employer":"Eyewax","email":"[email protected]","city":"Why","state":"KY"}
{"index":{"_id":"183"}}
{"account_number":183,"balance":14223,"firstname":"Hudson","lastname":"English","age":26,"gender":"F","address":"823 Herkimer Place","employer":"Xinware","email":"[email protected]","city":"Robbins","state":"ND"}
{"index":{"_id":"188"}}
{"account_number":188,"balance":41504,"firstname":"Tia","lastname":"Miranda","age":24,"gender":"F","address":"583 Ainslie Street","employer":"Jasper","email":"[email protected]","city":"Summerset","state":"UT"}
{"index":{"_id":"190"}}
{"account_number":190,"balance":3150,"firstname":"Blake","lastname":"Davidson","age":30,"gender":"F","address":"636 Diamond Street","employer":"Quantasis","email":"[email protected]","city":"Crumpler","state":"KY"}
{"index":{"_id":"195"}}
{"account_number":195,"balance":5025,"firstname":"Kaye","lastname":"Gibson","age":31,"gender":"M","address":"955 Hopkins Street","employer":"Zork","email":"[email protected]","city":"Ola","state":"WY"}
{"index":{"_id":"203"}}
{"account_number":203,"balance":21890,"firstname":"Eve","lastname":"Wyatt","age":33,"gender":"M","address":"435 Furman Street","employer":"Assitia","email":"[email protected]","city":"Jamestown","state":"MN"}
{"index":{"_id":"208"}}
{"account_number":208,"balance":40760,"firstname":"Garcia","lastname":"Hess","age":26,"gender":"F","address":"810 Nostrand Avenue","employer":"Quiltigen","email":"[email protected]","city":"Brooktrails","state":"GA"}
{"index":{"_id":"210"}}
{"account_number":210,"balance":33946,"firstname":"Cherry","lastname":"Carey","age":24,"gender":"M","address":"539 Tiffany Place","employer":"Martgo","email":"[email protected]","city":"Fairacres","state":"AK"}
{"index":{"_id":"215"}}
{"account_number":215,"balance":37427,"firstname":"Copeland","lastname":"Solomon","age":20,"gender":"M","address":"741 McDonald Avenue","employer":"Recognia","email":"[email protected]","city":"Edmund","state":"ME"}
{"index":{"_id":"222"}}
{"account_number":222,"balance":14764,"firstname":"Rachelle","lastname":"Rice","age":36,"gender":"M","address":"333 Narrows Avenue","employer":"Enaut","email":"[email protected]","city":"Wright","state":"AZ"}
{"index":{"_id":"227"}}
{"account_number":227,"balance":19780,"firstname":"Coleman","lastname":"Berg","age":22,"gender":"M","address":"776 Little Street","employer":"Exoteric","email":"[email protected]","city":"Eagleville","state":"WV"}
{"index":{"_id":"234"}}
{"account_number":234,"balance":44207,"firstname":"Betty","lastname":"Hall","age":37,"gender":"F","address":"709 Garfield Place","employer":"Miraclis","email":"[email protected]","city":"Bendon","state":"NY"}
{"index":{"_id":"239"}}
{"account_number":239,"balance":25719,"firstname":"Chang","lastname":"Boyer","age":36,"gender":"M","address":"895 Brigham Street","employer":"Qaboos","email":"[email protected]","city":"Belgreen","state":"NH"}
{"index":{"_id":"241"}}
{"account_number":241,"balance":25379,"firstname":"Schroeder","lastname":"Harrington","age":26,"gender":"M","address":"610 Tapscott Avenue","employer":"Otherway","email":"[email protected]","city":"Ebro","state":"TX"}
{"index":{"_id":"246"}}
{"account_number":246,"balance":28405,"firstname":"Katheryn","lastname":"Foster","age":21,"gender":"F","address":"259 Kane Street","employer":"Quantalia","email":"[email protected]","city":"Bath","state":"TX"}
{"index":{"_id":"253"}}
{"account_number":253,"balance":20240,"firstname":"Melissa","lastname":"Gould","age":31,"gender":"M","address":"440 Fuller Place","employer":"Buzzopia","email":"[email protected]","city":"Lumberton","state":"MD"}
{"index":{"_id":"258"}}
{"account_number":258,"balance":5712,"firstname":"Lindsey","lastname":"Hawkins","age":37,"gender":"M","address":"706 Frost Street","employer":"Enormo","email":"[email protected]","city":"Gardners","state":"AK"}
{"index":{"_id":"260"}}
{"account_number":260,"balance":2726,"firstname":"Kari","lastname":"Skinner","age":30,"gender":"F","address":"735 Losee Terrace","employer":"Singavera","email":"[email protected]","city":"Rushford","state":"WV"}
{"index":{"_id":"265"}}
{"account_number":265,"balance":46910,"firstname":"Marion","lastname":"Schneider","age":26,"gender":"F","address":"574 Everett Avenue","employer":"Evidends","email":"[email protected]","city":"Maplewood","state":"WY"}
{"index":{"_id":"272"}}
{"account_number":272,"balance":19253,"firstname":"Lilly","lastname":"Morgan","age":25,"gender":"F","address":"689 Fleet Street","employer":"Biolive","email":"[email protected]","city":"Sunbury","state":"OH"}
{"index":{"_id":"277"}}
{"account_number":277,"balance":29564,"firstname":"Romero","lastname":"Lott","age":31,"gender":"M","address":"456 Danforth Street","employer":"Plasto","email":"[email protected]","city":"Vincent","state":"VT"}
{"index":{"_id":"284"}}
{"account_number":284,"balance":22806,"firstname":"Randolph","lastname":"Banks","age":29,"gender":"M","address":"875 Hamilton Avenue","employer":"Caxt","email":"[email protected]","city":"Crawfordsville","state":"WA"}
{"index":{"_id":"289"}}
{"account_number":289,"balance":7798,"firstname":"Blair","lastname":"Church","age":29,"gender":"M","address":"370 Sutton Street","employer":"Cubix","email":"[email protected]","city":"Nile","state":"NH"}
{"index":{"_id":"291"}}
{"account_number":291,"balance":19955,"firstname":"Lynn","lastname":"Pollard","age":40,"gender":"F","address":"685 Pierrepont Street","employer":"Slambda","email":"[email protected]","city":"Mappsville","state":"ID"}
{"index":{"_id":"296"}}
{"account_number":296,"balance":24606,"firstname":"Rosa","lastname":"Oliver","age":34,"gender":"M","address":"168 Woodbine Street","employer":"Idetica","email":"[email protected]","city":"Robinson","state":"WY"}
{"index":{"_id":"304"}}
{"account_number":304,"balance":28647,"firstname":"Palmer","lastname":"Clark","age":35,"gender":"M","address":"866 Boulevard Court","employer":"Maximind","email":"[email protected]","city":"Avalon","state":"NH"}
{"index":{"_id":"309"}}
{"account_number":309,"balance":3830,"firstname":"Rosemarie","lastname":"Nieves","age":30,"gender":"M","address":"206 Alice Court","employer":"Zounds","email":"[email protected]","city":"Ferney","state":"AR"}
{"index":{"_id":"311"}}
{"account_number":311,"balance":13388,"firstname":"Vinson","lastname":"Ballard","age":23,"gender":"F","address":"960 Glendale Court","employer":"Gynk","email":"[email protected]","city":"Fairforest","state":"WY"}
{"index":{"_id":"316"}}
{"account_number":316,"balance":8214,"firstname":"Anita","lastname":"Ewing","age":32,"gender":"M","address":"396 Lombardy Street","employer":"Panzent","email":"[email protected]","city":"Neahkahnie","state":"WY"}
{"index":{"_id":"323"}}
{"account_number":323,"balance":42230,"firstname":"Chelsea","lastname":"Gamble","age":34,"gender":"F","address":"356 Dare Court","employer":"Isosphere","email":"[email protected]","city":"Dundee","state":"MD"}
{"index":{"_id":"328"}}
{"account_number":328,"balance":12523,"firstname":"Good","lastname":"Campbell","age":27,"gender":"F","address":"438 Hicks Street","employer":"Gracker","email":"[email protected]","city":"Marion","state":"CA"}
{"index":{"_id":"330"}}
{"account_number":330,"balance":41620,"firstname":"Yvette","lastname":"Browning","age":34,"gender":"F","address":"431 Beekman Place","employer":"Marketoid","email":"[email protected]","city":"Talpa","state":"CO"}
{"index":{"_id":"335"}}
{"account_number":335,"balance":35433,"firstname":"Vera","lastname":"Hansen","age":24,"gender":"M","address":"252 Bushwick Avenue","employer":"Zanilla","email":"[email protected]","city":"Manila","state":"TN"}
{"index":{"_id":"342"}}
{"account_number":342,"balance":33670,"firstname":"Vivian","lastname":"Wells","age":36,"gender":"M","address":"570 Cobek Court","employer":"Nutralab","email":"[email protected]","city":"Fontanelle","state":"OK"}
{"index":{"_id":"347"}}
{"account_number":347,"balance":36038,"firstname":"Gould","lastname":"Carson","age":24,"gender":"F","address":"784 Pulaski Street","employer":"Mobildata","email":"[email protected]","city":"Goochland","state":"MI"}
{"index":{"_id":"354"}}
{"account_number":354,"balance":21294,"firstname":"Kidd","lastname":"Mclean","age":22,"gender":"M","address":"691 Saratoga Avenue","employer":"Ronbert","email":"[email protected]","city":"Tioga","state":"ME"}
{"index":{"_id":"359"}}
{"account_number":359,"balance":29927,"firstname":"Vanessa","lastname":"Harvey","age":28,"gender":"F","address":"679 Rutledge Street","employer":"Zentime","email":"[email protected]","city":"Williston","state":"IL"}
{"index":{"_id":"361"}}
{"account_number":361,"balance":23659,"firstname":"Noreen","lastname":"Shelton","age":36,"gender":"M","address":"702 Tillary Street","employer":"Medmex","email":"[email protected]","city":"Derwood","state":"NH"}
{"index":{"_id":"366"}}
{"account_number":366,"balance":42368,"firstname":"Lydia","lastname":"Cooke","age":31,"gender":"M","address":"470 Coleman Street","employer":"Comstar","email":"[email protected]","city":"Datil","state":"TN"}
{"index":{"_id":"373"}}
{"account_number":373,"balance":9671,"firstname":"Simpson","lastname":"Carpenter","age":21,"gender":"M","address":"837 Horace Court","employer":"Snips","email":"[email protected]","city":"Tolu","state":"MA"}
{"index":{"_id":"378"}}
{"account_number":378,"balance":27100,"firstname":"Watson","lastname":"Simpson","age":36,"gender":"F","address":"644 Thomas Street","employer":"Wrapture","email":"[email protected]","city":"Keller","state":"TX"}
{"index":{"_id":"380"}}
{"account_number":380,"balance":35628,"firstname":"Fernandez","lastname":"Reid","age":33,"gender":"F","address":"154 Melba Court","employer":"Cosmosis","email":"[email protected]","city":"Boyd","state":"NE"}
{"index":{"_id":"385"}}
{"account_number":385,"balance":11022,"firstname":"Rosalinda","lastname":"Valencia","age":22,"gender":"M","address":"933 Lloyd Street","employer":"Zoarere","email":"[email protected]","city":"Waverly","state":"GA"}
{"index":{"_id":"392"}}
{"account_number":392,"balance":31613,"firstname":"Dotson","lastname":"Dean","age":35,"gender":"M","address":"136 Ford Street","employer":"Petigems","email":"[email protected]","city":"Chical","state":"SD"}
{"index":{"_id":"397"}}
{"account_number":397,"balance":37418,"firstname":"Leonard","lastname":"Gray","age":36,"gender":"F","address":"840 Morgan Avenue","employer":"Recritube","email":"[email protected]","city":"Edenburg","state":"AL"}
{"index":{"_id":"400"}}
{"account_number":400,"balance":20685,"firstname":"Kane","lastname":"King","age":21,"gender":"F","address":"405 Cornelia Street","employer":"Tri@Tribalog","email":"kaneking@[email protected]","city":"Gulf","state":"VT"}
{"index":{"_id":"405"}}
{"account_number":405,"balance":5679,"firstname":"Strickland","lastname":"Fuller","age":26,"gender":"M","address":"990 Concord Street","employer":"Digique","email":"[email protected]","city":"Southmont","state":"NV"}
{"index":{"_id":"412"}}
{"account_number":412,"balance":27436,"firstname":"Ilene","lastname":"Abbott","age":26,"gender":"M","address":"846 Vine Street","employer":"Typhonica","email":"[email protected]","city":"Cedarville","state":"VT"}
{"index":{"_id":"417"}}
{"account_number":417,"balance":1788,"firstname":"Wheeler","lastname":"Ayers","age":35,"gender":"F","address":"677 Hope Street","employer":"Fortean","email":"[email protected]","city":"Ironton","state":"PA"}
{"index":{"_id":"424"}}
{"account_number":424,"balance":36818,"firstname":"Tracie","lastname":"Gregory","age":34,"gender":"M","address":"112 Hunterfly Place","employer":"Comstruct","email":"[email protected]","city":"Onton","state":"TN"}
{"index":{"_id":"429"}}
{"account_number":429,"balance":46970,"firstname":"Cantu","lastname":"Lindsey","age":31,"gender":"M","address":"404 Willoughby Avenue","employer":"Inquala","email":"[email protected]","city":"Cowiche","state":"IA"}
{"index":{"_id":"431"}}
{"account_number":431,"balance":13136,"firstname":"Laurie","lastname":"Shaw","age":26,"gender":"F","address":"263 Aviation Road","employer":"Zillanet","email":"[email protected]","city":"Harmon","state":"WV"}
{"index":{"_id":"436"}}
{"account_number":436,"balance":27585,"firstname":"Alexander","lastname":"Sargent","age":23,"gender":"M","address":"363 Albemarle Road","employer":"Fangold","email":"[email protected]","city":"Calpine","state":"OR"}
{"index":{"_id":"443"}}
{"account_number":443,"balance":7588,"firstname":"Huff","lastname":"Thomas","age":23,"gender":"M","address":"538 Erskine Loop","employer":"Accufarm","email":"[email protected]","city":"Corinne","state":"AL"}
{"index":{"_id":"448"}}
{"account_number":448,"balance":22776,"firstname":"Adriana","lastname":"Mcfadden","age":35,"gender":"F","address":"984 Woodside Avenue","employer":"Telequiet","email":"[email protected]","city":"Darrtown","state":"WI"}
{"index":{"_id":"450"}}
{"account_number":450,"balance":2643,"firstname":"Bradford","lastname":"Nielsen","age":25,"gender":"M","address":"487 Keen Court","employer":"Exovent","email":"[email protected]","city":"Hamilton","state":"DE"}
{"index":{"_id":"455"}}
{"account_number":455,"balance":39556,"firstname":"Lynn","lastname":"Tran","age":36,"gender":"M","address":"741 Richmond Street","employer":"Optyk","email":"[email protected]","city":"Clinton","state":"WV"}
{"index":{"_id":"462"}}
{"account_number":462,"balance":10871,"firstname":"Calderon","lastname":"Day","age":27,"gender":"M","address":"810 Milford Street","employer":"Cofine","email":"[email protected]","city":"Kula","state":"OK"}
{"index":{"_id":"467"}}
{"account_number":467,"balance":6312,"firstname":"Angelica","lastname":"May","age":32,"gender":"F","address":"384 Karweg Place","employer":"Keeg","email":"[email protected]","city":"Tetherow","state":"IA"}
{"index":{"_id":"474"}}
{"account_number":474,"balance":35896,"firstname":"Obrien","lastname":"Walton","age":40,"gender":"F","address":"192 Ide Court","employer":"Suremax","email":"[email protected]","city":"Crucible","state":"UT"}
{"index":{"_id":"479"}}
{"account_number":479,"balance":31865,"firstname":"Cameron","lastname":"Ross","age":40,"gender":"M","address":"904 Bouck Court","employer":"Telpod","email":"[email protected]","city":"Nord","state":"MO"}
{"index":{"_id":"481"}}
{"account_number":481,"balance":20024,"firstname":"Lina","lastname":"Stanley","age":33,"gender":"M","address":"361 Hanover Place","employer":"Strozen","email":"[email protected]","city":"Wyoming","state":"NC"}
{"index":{"_id":"486"}}
{"account_number":486,"balance":35902,"firstname":"Dixie","lastname":"Fuentes","age":22,"gender":"F","address":"991 Applegate Court","employer":"Portico","email":"[email protected]","city":"Salix","state":"VA"}
{"index":{"_id":"493"}}
{"account_number":493,"balance":5871,"firstname":"Campbell","lastname":"Best","age":24,"gender":"M","address":"297 Friel Place","employer":"Fanfare","email":"[email protected]","city":"Kidder","state":"GA"}
{"index":{"_id":"498"}}
{"account_number":498,"balance":10516,"firstname":"Stella","lastname":"Hinton","age":39,"gender":"F","address":"649 Columbia Place","employer":"Flyboyz","email":"[email protected]","city":"Crenshaw","state":"SC"}

View Code

9. reindex 重索引

Reindex API /_reindex 讓我們可以將一個索引中的數據重索引到另一個索引中（拷貝），要求源索引的_source 是開啟的。目標索引的setting 、mapping 信息與源索引無關。

什麽時候需要重索引？

即當需要做數據的拷貝的時候

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}

重索引要考慮的一個問題：目標索引中存在源索引中的數據，這些數據的version如何處理。

1、如果沒有指定version_type 或指定為 internal，則會是采用目標索引中的版本，重索引過程中，執行的就是新增、更新操作。

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "internal"
  }
}

2、如果想使用源索引中的版本來進行版本控制更新，則設置 version_type 為extenal。重索引操作將寫入不存在的，更新舊版本的數據。

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "external"
  }
}

如果你只想從源索引中復制目標索引中不存在的文檔數據，可以指定 op_type 為 create 。此時存在的文檔將觸發版本沖突（會導致放棄操作），可設置“conflicts”: “proceed“，跳過繼續

POST _reindex
{
  "conflicts": "proceed",
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "op_type": "create"
  }
}

你也可以只索引源索引的一部分數據，通過 type 或查詢來指定你需要的數據

POST _reindex
{
  "source": {
    "index": "twitter",
    "type": "_doc",
    "query": {
      "term": {
        "user": "kimchy"
      }
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}

可以從多個源獲取數據

POST _reindex
{
  "source": {
    "index": ["twitter", "blog"],
    "type": ["_doc", "post"]
  },
  "dest": {
    "index": "all_together"
  }
}

可以限定文檔數量

POST _reindex
{
  "size": 10000,
  "source": {
    "index": "twitter",
    "sort": { "date": "desc" }
  },
  "dest": {
    "index": "new_twitter"
  }
}

可以選擇復制源文檔的哪些字段

POST _reindex
{
  "source": {
    "index": "twitter",
    "_source": ["user", "_doc"]
  },
  "dest": {
    "index": "new_twitter"
  }
}

可以用script來改變文檔

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "external"
  },
  "script": {
    "source": "if (ctx._source.foo == ‘bar‘) {ctx._version++; ctx._source.remove(‘foo‘)}",
    "lang": "painless"
  }
}

可以指定路由值把文檔放到哪個分片上

POST _reindex
{
  "source": {
    "index": "source",
    "query": {
      "match": {
        "company": "cat"
      }
    }
  },
  "dest": {
    "index": "dest",
    "routing": "=cat"
  }
}

從遠程源復制

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "username": "user",
      "password": "pass"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}

通過_task 來查詢執行狀態

GET _tasks?detailed=true&actions=*reindex

10. refresh

對於索引、更新、刪除操作如果想操作完後立馬重刷新可見，可帶上refresh參數

PUT /test/_doc/1?refresh
{"test": "test"}
PUT /test/_doc/2?refresh=true
{"test": "test"}

refresh 可選值說明

未給值或=true，則立馬會重刷新讀索引。
=false ，相當於沒帶refresh 參數，遵循內部的定時刷新。
=wait_for ，登記等待刷新，當登記的請求數達到index.max_refresh_listeners 參數設定的值時(defaults to 1000)，將觸發重刷新。

三、路由詳解

技術分享圖片

1. 集群組成

第一個節點啟動

技術分享圖片

說明：首先啟動的一定是主節點，主節點存儲的是集群的元數據信息

Node2啟動

技術分享圖片

說明：

Node2節點啟動之前會配置集群的名稱Cluster-name：ess，然後配置可以作為主節點的ip地址信息discovery.zen.ping.unicast.hosts: [“10.0.1.11",“10.0.1.12"]，配置自己的ip地址networ.host: 10.0.1.12；

Node2啟動的過程中會去找到主節點Node1告訴Node1我要加入到集群裏面了，主節點Node1接收到請求以後看Node2是否滿足加入集群的條件，如果滿足就把node2的ip地址加入的元信息裏面，然後廣播給集群中的其他節點有

新節點加入，並把最新的元信息發送給其他的節點去更新

Node3..NodeN加入

技術分享圖片

說明：集群中的所有節點的元信息都是和主節點一致的，因為一旦有新的節點加入進來，主節點會通知其他的節點同步元信息

2. 在集群中創建索引的流程

技術分享圖片

3. 有索引的集群

技術分享圖片

4. 集群有節點出現故障，如主節點掛了，會重新選擇主節點

技術分享圖片

5. 在集群中索引文檔

技術分享圖片

索引文檔的步驟：
1、node2計算文檔的路由值得到文檔存放的分片（假定路由選定的是分片0）。
2、將文檔轉發給分片0(P0)的主分片節點 node1。
3、node1索引文檔，同步給副本（R0）節點node3索引文檔。
4、node1向node2反饋結果
5、node2作出響應

6. 文檔是如何路由的

文檔該存到哪個分片上？
決定文檔存放到哪個分片上就是文檔路由。ES中通過下面的計算得到每個文檔的存放分片：

shard = hash(routing) % number_of_primary_shards

參數說明：

routing 是用來進行hash計算的路由值，默認是使用文檔id值。我們可以在索引文檔時通過routing參數指定別的路由值

number_of_primary_shards：創建索引時指定的主分片數

POST twitter/_doc?routing=kimchy
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

在索引、刪除、更新、查詢中都可以使用routing參數（可多值）指定操作的分片。

創建索引時強制要求給定路由值：

PUT my_index2
{
  "mappings": {
    "_doc": {
      "_routing": {
        "required": true 
      }
    }
  }
}

7. 在集群中進行搜索

技術分享圖片

搜索的步驟：如要搜索索引 s0
1、node2解析查詢。
2、node2將查詢發給索引s0的分片/副本（R1,R2,R0）節點
3、各節點執行查詢，將結果發給Node2
4、Node2合並結果，作出響應。

8. Master節點的工作是什麽？

1. 存儲集群的元信息，如集群名稱、集群中的節點

2. 轉發創建索引和索引文檔的請求

3. 和其他的節點進行通信，告訴其他節點有新的節點加入等

elasticsearch系列三：索引詳解（分詞器、文檔管理、路由詳解）

ces com dex 合並 pda ams 最新 case dbi 一、分詞器 1. 認識分詞器 1.1 Analyzer 分析器在ES中一個Analyzer 由下面三種組件組合而成： character filter ：字符過濾器，對文本進行字符過濾處理，

ElasticSearch優化系列三：索引過程

大家可能會遇到索引資料比較慢的過程。其實明白索引的原理就可以有針對性的進行優化。ES索引的過程到相對Lucene的索引過程多了分散式資料的擴充套件，而這ES主要是用tranlog進行各節點之間的資料平衡。所以從上我可以通過索引的settings進行第一優化： "index.translog.flus

Silverlight & Blend動畫設計系列三：縮放動畫（ScaleTransform）

技術 sel transform 離開 src 比例復制代碼創建對象 keyframes 在Silverlight的動畫框架中，ScaleTransform類提供了在二維空間中的坐標內進行縮放操作，通過ScaleTransform可以在水平或垂直方向的縮放和拉伸對象，以

使用 PySide2 開發 Maya 外掛系列三：qt語言國際化（internationalization）

使用 PySide2 開發 Maya 外掛系列三：qt語言國際化（internationalization）前言：這是 qt for python 的語言國際化，基於 UI 的，python 也有自身的語言國際化，兩者是不同的。先來看最終效果：前期準備：這次建立一個 main wind

NLP --- 分詞詳解（分詞的歷史）

前面的內容我們以演算法為主，主要講解了HMM和CRF演算法，為了解決HMM的問題，使用了EM演算法，但是整個HMM演算法是產生式模型的，計算量大和複雜，因此為了解決這個問題，我們引入了CRF演算法，該演算法是借鑑了最大熵模型的演算法，然後在HMM的基礎上加入了概率圖模型和團的概念，使的最後模型稱

《從零開始學Swift》學習筆記（Day 57）——Swift編碼規範之凝視規範：文件凝視、文檔凝視、代碼凝視、使用地標凝視

精品 -type mil 顯示 clas ber ansi tex text 原創文章。歡迎轉載。轉載請註明：關東升的博客前面說到Swift凝視的語法有兩種：單行凝視（//）和多行凝視（/*...*/）。這裏來介紹一下他們的使用規範。 1、文件凝視文件凝視就在每個文

Elasticsearch入門之從零開始安裝ik分詞器

gpo article terms n) rm -rf 從零開始系列 pack 默認起因需要在ES中使用聚合進行統計分析，但是聚合字段值為中文，ES的默認分詞器對於中文支持非常不友好：會把完整的中文詞語拆分為一系列獨立的漢字進行聚合，顯然這並不是我的初衷。我們來看個

elasticsearch-手動設定_mapping中欄位型別及分詞器-ES5.X與ES6.X區別

【問題描述】：手動設定mapping，給指定欄位設定是否進行分詞，以及分詞器型別。環境資訊：學習elasticsearch時，案例使用的是5.x版本，自己本地使用的是6.x版本。案例如下，5.x版本編輯：PUT /website{ "mappings": { "art

Elasticsearch基於docker叢集搭建以及安裝ik分詞器

由於機器沒這麼多，所以用docker模擬真正的叢集搭建。 1、準備工作 1-1、準備docker環境：使用yum安裝docker: yum install -y docker-io 安裝完成後，開啟docker: systemctl start docker; 檢視d

elasticsearch系列四：搜尋詳解（搜尋API、Query DSL）

{ "took": 60, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 1000, "m

深入理解JAVA I/O系列三：字符流詳解

buffer 情況二進制文件感到復制代碼使用範圍轉換 fileread 方式字符流為何存在既然字節流提供了能夠處理任何類型的輸入/輸出操作的功能，那為什麽還要存在字符流呢？容我慢慢道來，字節流不能直接操作Unicode字符，因為一個字符有兩個字節，字節流一次只

Linux Rootkit系列三：例項詳解 Rootkit 必備的基本功能

前言鑑於筆者知識能力上的不足，如有疏忽，歡迎糾正。測試建議：不要在物理機測試！不要在物理機測試！不要在物理機測試！概要在上一篇文章中筆者詳細地闡述了基於直接修改系統呼叫表（即 sys_call_table /ia32_sys_call_

RabbitMQ系列教程之三：發布/訂閱（Publish/Subscribe）

mqc 標題整合參數 cti 事情 return 控制臺 run （本教程是使用Net客戶端，也就是針對微軟技術平臺的）在前一個教程中，我們創建了一個工作隊列。工作隊列背後的假設是每個任務會被交付給一個【工人】。在這一部分我們將做一些完全不同的事情--我們將向多個

elasticsearch系列一：elasticsearch（ES簡介、安裝&配置、集成Ikanalyzer）

ins 表示吞吐量 search 工作 use art tcp傳輸 .net 一、ES簡介 1. ES是什麽？ Elasticsearch 是一個開源的搜索引擎，建立在全文搜索引擎庫 Apache Lucene 基礎之上用 Java 編寫的，它的內部使用 Lucene

elasticsearch系列八：ES 集群管理（集群規劃、集群搭建、集群管理）

記得文件如果 cse init host 網絡隔離也有沒有一、集群規劃搭建一個集群我們需要考慮如下幾個問題： 1. 我們需要多大規模的集群？ 2. 集群中的節點角色如何分配？ 3. 如何避免腦裂問題？ 4. 索引應該設置多少個分片？ 5. 分片應該設置幾個副本？

ELK系列三：Elasticsearch的簡單使用和配置文件簡介

stat 創建索引 prop creat .com 索引 mapping 圖片 keyword 1、定義模板創建索引：首先定義好一個模板的例子 { "order":14, "template":"ids-1", "state":"open", "set

Elasticsearch入門三：Elasticsearch基本用法-增刪改查（譯）

title: Elasticsearch入門三：Elasticsearch基本用法-增刪改查（譯） date: 2018-11-01 11:00:00 tags: Elasticsearch 原文地址： https://www.elastic.co/guide/en/elast

Elasticsearch 系列指南（三）——整合ik分詞器

{ "tokens": [ { "token": "聯", "start_offset": 0, "end_offset": 1, "type": "<IDEOGRAPHIC>

Elasticsearch Java client（ES Client 簡介、Java REST Client、Java Client、Spring Data Elasticsearch） elasticsearch系列七：ES Java客戶端-Elasticsearch Java client（E

elasticsearch系列七：ES Java客戶端-Elasticsearch Java client（ES Client 簡介、Java REST Client、Java Client、Spring Data Elasticsearch）一、ES Client 簡介 1. ES是一個服務

mysql索引之三：索引使用注意規則（索引失效--存在索引但不使用索引）*

使用索引時，有以下一些技巧和注意事項： (1)越小的資料型別通常更好：越小的資料型別通常在磁碟、記憶體和CPU快取中都需要更少的空間，處理起來更快。(2)簡單的資料型別更好：整型資料比起字元，處理開銷更小，因為字串的比較更復雜。在MySQL中，應該用內建的日期和時間資料型別，而不是用字串來儲存時間；以及用整

elasticsearch系列三：索引詳解（分詞器、文檔管理、路由詳解）

一、分詞器

1. 認識分詞器

2. 內建的character filter

3. 內建的Tokenizer

4. 內建的Token Filter

5. 內建的Analyzer

6. Analyzer的使用順序

二、文檔管理

1. 新建文檔

2. 獲取單個文檔

3. 獲取多個文檔 _mget

4. 刪除文檔

5. 更新文檔

6.Scripted update 通過腳本來更新文檔

7. 通過條件查詢來更新文檔

8. 批量操作

9. reindex 重索引

10. refresh

三、路由詳解

1. 集群組成

2. 在集群中創建索引的流程

3. 有索引的集群

4. 集群有節點出現故障，如主節點掛了，會重新選擇主節點

5. 在集群中索引文檔

6. 文檔是如何路由的

7. 在集群中進行搜索

8. Master節點的工作是什麽？

相關推薦