19.ELK實時日誌分析平臺之Elasticsearch REST API簡介
Elasticsearch提供了一系列RESTful的API,覆蓋瞭如下功能:
- 檢查叢集、節點、索引的健康度、狀態和統計
- 管理叢集、節點、索引的資料及元資料
- 對索引進行CRUD操作及查詢操作
- 執行其他高階操作如分頁、排序、過濾等。
叢集資訊
使用_cat API可以查詢叢集健康,比如:
$ curl 'localhost:9200/_cat/health?v'
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1458099890 11:44:50 MegviiQA green 1 1 0 0 0 0 0 0 - 100.0%
使用不提供v引數可以查詢簡要資訊:
$ curl 'localhost:9200/_cat/health'
1458099895 11:44:55 MegviiQA green 1 1 0 0 0 0 0 0 - 100.0%
關於各個欄位的解釋,可以使用help引數,比如:
$ curl 'localhost:9200/_cat/health?help'
epoch | t,time | seconds since 1970-01-01 00:00:00
timestamp | ts,hms,hhmmss | time in HH:MM:SS
cluster | cl | cluster name
status | st | health status
node.total | nt,nodeTotal | total number of nodes
node.data | nd,nodeData | number of nodes that can store data
shards | t,sh,shards.total,shardsTotal | total number of shards
pri | p,shards.primary,shardsPrimary | number of primary shards
relo | r,shards.relocating,shardsRelocating | number of relocating nodes
init | i,shards.initializing,shardsInitializing | number of initializing nodes
unassign | u,shards.unassigned,shardsUnassigned | number of unassigned shards
pending_tasks | pt,pendingTasks | number of pending tasks
max_task_wait_time | mtwt,maxTaskWaitTime | wait time of longest task pending
active_shards_percent | asp,activeShardsPercent | active number of shards in percent
叢集狀態的顏色分紅、黃、綠三種,紅色需要修復資料,黃色表示某些replica尚未被分配到其他節點但不影響整個叢集的功能,綠色表明整個叢集的功能正常。
查詢叢集的節點資訊可以使用如下介面:
$ curl 'localhost:9200/_cat/nodes?v'
host ip heap.percent ram.percent load node.role master name
127.0.0.1 127.0.0.1 4 88 0.04 d * QA-103.6-elasticsearch
操作索引
羅列所有索引資訊的介面如下:
$ curl 'localhost:9200/_cat/indices?v'
health status index pri rep docs.count docs.deleted store.size pri.store.size
上面的輸出,說明我們目前還沒有建立過任何索引。
可以使用PUT或POST方法建立索引:
$ curl -XPUT 'localhost:9200/customer'
{"acknowledged":true}
$ curl -XPOST 'localhost:9200/seller'
{"acknowledged":true}
$ curl 'localhost:9200/_cat/indices?v'
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open customer 5 1 0 0 650b 650b
yellow open seller 5 1 0 0 650b 650b
從上述輸出中可知,customer這個索引,有5個primary shards,有1個replica(預設的),其中不包含任何內容(document的數量是0)。新建索引的狀態是yellow,因為預設為索引準備了一個備份但是沒有另一個節點去分配,如果叢集中加入了另一個節點,replica分配完成後,索引的狀態就會使綠色的了。
向索引中新增資料的方法如下:
$ curl -XPUT 'localhost:9200/customer/external/1?pretty' -d '
> {
> "name":"lmz"
> }'
{
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 1,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"created" : true
}
注意:索引一個document的時候不需要先建立索引,可以直接新增資料。
查詢一條資料的方法如下:
$ curl -XGET 'localhost:9200/customer/external/1?pretty'
{
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 1,
"found" : true,
"_source" : {
"name" : "lmz"
}
}
刪除索引可以使用如下介面:
$ curl -XDELETE 'localhost:9200/customer?pretty'
{
"acknowledged" : true
}
查詢索引可以看到索引成功刪除:
$ curl -XGET 'localhost:9200/_cat/indices?v'
health status index pri rep docs.count docs.deleted store.size pri.store.size
更新資料
為一個具體的(index+type+id)document賦值之後,如果使用相同的index+type+id對其進行PUT操作,會更新該document的值(version的值會加一)。
新增一個document的時候,也可以不指定id,這時Elasticsearch會隨機生成一個id,這種情況必須使用POST方法,而且生成的元素的id不一定是數字形式,比如:
$ curl -XPOST 'localhost:9200/customer/external?pretty' -d '
> {
> "name": "Jane Doe"
> }'
{
"_index" : "customer",
"_type" : "external",
"_id" : "AVN-lf9kG-SL-FxTq4sA",
"_version" : 1,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"created" : true
}
更新也可以使用專門的update介面,注意Elasticsearch的更新並不是一個原地操作,它只是刪除原來的document然後建立一個新的而已。
$ curl -XPOST 'localhost:9200/customer/external/1/_update?pretty' -d '
{
"doc":{"name":"mars loo"}
}'
{
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 2,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
}
}
也可以在更新的過程中新增新的欄位:
$ curl -XPOST 'localhost:9200/customer/external/1/_update?pretty' -d '
{
"doc":{"name": "mars loo", "age": 25}
}'
{
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 3,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
}
}
更新操作也可以使用指令碼:
~$ curl -XPOST 'localhost:8200/customer/external/1/_update?pretty' -d '
{
"script":"ctx._source.doc.age += 5"
}'
{
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
}
}
其中ctx._source
表示當前待更新的document。
刪除一個document可以使用如下介面:
$ curl -XDELETE 'localhost:9200/customer/external/1?pretty'
{
"found" : true,
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 4,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
}
}
查詢這個document確認found欄位為false:
$ curl -XGET 'localhost:9200/customer/external/1?pretty'
{
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"found" : false
}
批量操作
Elasticsearch提供了進行批量操作的_bulk API,能夠方便的進行批量處理,例如一次性索引兩個document:
$ curl -XPOST 'localhost:9200/customer/external/_bulk?pretty' -d'
> {"index":{"_id":1}}
> {"name":"mars"}
> {"index":{"_id":2}}
> {"name":"suson"}
> '
{
"took" : 134,
"errors" : false,
"items" : [ {
"index" : {
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 1,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"status" : 201
}
}, {
"index" : {
"_index" : "customer",
"_type" : "external",
"_id" : "2",
"_version" : 1,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"status" : 201
}
} ]
}
在一次操作中更新_id為1的document,刪除_id為2的document:
$ curl -XPOST 'localhost:9200/customer/external/_bulk?pretty' -d '
> {"update":{"_id":1}}
> {"doc":{"name":"mars loo", "age":25}}
> {"delete":{"_id":2}}
> '
{
"took" : 208,
"errors" : false,
"items" : [ {
"update" : {
"_index" : "customer",
"_type" : "external",
"_id" : "1",
"_version" : 2,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"status" : 200
}
}, {
"delete" : {
"_index" : "customer",
"_type" : "external",
"_id" : "2",
"_version" : 2,
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"status" : 200,
"found" : true
}
} ]
}
批量操作會按照入參順序依次執行每個動作,即使某個動作失敗了,也會繼續執行剩下的動作。在最後的返回結果中,Elasticsearch會按照入參的順序詳細地給出每個動作的執行情況。
如果覺得我的文章對您有幫助,歡迎關注我(CSDN:Mars Loo的部落格)或者為這篇文章點贊,謝謝!