穀粒商城--分散式高階篇P102~P128
由於學習的時間也比較少，只有週六週末才有時間出來學習總結，所以一篇一篇慢慢更新吧，本次總結內容為Elasticsearch（相關內容：kibana，es，nginx，ik分詞器）
【穀粒商城--分散式基礎篇P1~P27】: https://blog.csdn.net/Empire_ing/article/details/118860147
【穀粒商城--分散式基礎篇P28~P101】https://mp.weixin.qq.com/s/5kvXjLNyVn-GBhNMWyJdpg
@
目錄
1.ElasticSearch與kibana
內容概況：es、kibana、ik相關軟體於centos中docker的下載與配置使用。本次安裝最大感悟就是docker太好用了，真的好強。
之前也寫過一篇關於es文章例項：
SpringBoot整合es實現高亮搜尋：https://www.cnblogs.com/meditation5201314/p/14801585.html
1.1.基本概念
索引index：類似於資料庫
型別type：類似於庫中的表
文件document：類似於表的每一行
方法/屬性：類似於每一行的欄位
1.2 安裝es與kibana
1. ES訪問地址：http://公網IP:9200/
# 1、下載包
docker pull elasticsearch:7.4.2
# 2、配置es檔案對映
# 將docker裡的目錄掛載到linux的/mydata目錄中
# 修改/mydata就可以改掉docker裡的
mkdir -p /mydata/elasticsearch/config
mkdir -p /mydata/elasticsearch/data
# es可以被遠端任何機器訪問
echo "http.host: 0.0.0.0" >/mydata/elasticsearch/config/elasticsearch.yml
# 遞迴更改許可權，es需要訪問
chmod -R 777 /mydata/elasticsearch/
# 3、啟動es
# 9200是使用者互動埠 9300是叢集心跳埠 -e指定是單階段執行 -e指定佔用的記憶體大小，生產時可以設定32G
docker run --name elasticsearch -p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
-e ES_JAVA_OPTS="-Xms64m -Xmx512m" \
-v /mydata/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
-v /mydata/elasticsearch/data:/usr/share/elasticsearch/data \
-v /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
-d elasticsearch:7.4.2
# 4、設定開機啟動elasticsearch
docker update elasticsearch --restart=always
- kibana下載、配置、啟動
Kibana訪問地址：http://公網IP:5601/
# 1、下載包
docker pull kibana:7.4.2
# 2、啟動es（這裡填自己的IP地址）
docker run --name kibana -e ELASTICSEARCH_HOSTS=http://公網IP:9200 -p 5601:5601 -d kibana:7.4.2
# 3、設定開機啟動kibana
docker update kibana --restart=always
訪問地址：
檢視es的node節點：http://公網IP:9200/_cat/nodes
檢視es的helth健康狀態：http://公網IP:9200/_cat/health
檢視es的master主節點：http://公網IP:9200/_cat/master
檢視es的索引主節點：http://公網IP:9200/_cat/indices
1.3 Nginx安裝
docker run -p80:80 --name nginx -d nginx:1.10
mkdir -p /mydata/nginx/html
mkdir -p /mydata/nginx/logs
mkdir -p /mydata/nginx/conf
docker container cp nginx:/etc/nginx/* /mydata/nginx/conf/
#由於拷貝完成後會在config中存在一個nginx資料夾，所以需要將它的內容移動到conf中
mv /mydata/nginx/conf/nginx/* /mydata/nginx/conf/
rm -rf /mydata/nginx/conf/nginx
docker stop nginx
docker rm nginx
docker run -p 80:80 --name nginx \
-v /mydata/nginx/html:/usr/share/nginx/html \
-v /mydata/nginx/logs:/var/log/nginx \
-v /mydata/nginx/conf/:/etc/nginx \
-d nginx:1.10
docker update nginx --restart=always
1.4 Ik中文分詞器
在elasticsearch/plugins目錄下下載解壓ik分詞器後重啟即可
#下載ik分詞器
wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.4.2/elasticsearch-analysis-ik-7.4.2.zip
#解壓並重命名ik
unzip elasticsearch-analysis-ik-7.4.2.zip -d ik
chmod -R 777 plugins/ik
rm -rf elasticsearch-analysis-ik-7.4.2.zip #這裡要保證plugins目錄下純淨，否則啟動會報錯
docker restart elasticsearch
2.SpringBoot整合Es
2.1.環境配置
- ELK版本對應並去掉springboot中自帶的es包
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.4.2</version>
</dependency>
<properties>
<java.version>1.8</java.version>
<elasticsearch.version>7.4.2</elasticsearch.version>
</properties>
- Config配置
package com.empirefree.gulimall.search.config;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* @program: gulimall-search
* @description:
* @author: huyuqiao
* @create: 2021/08/01 13:54
*/
@Configuration
public class GulimallElasticSearchConfig {
public static final RequestOptions COMMON_OPTIONS;
static {
RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
// builder.addHeader("Authorization", "Bearer " + TOKEN);
// builder.setHttpAsyncResponseConsumerFactory(
// new HttpAsyncResponseConsumerFactory
// .HeapBufferedResponseConsumerFactory(30 * 1024 * 1024 * 1024));
COMMON_OPTIONS = builder.build();
}
@Bean
public RestHighLevelClient restHighLevelClient(){
RestClientBuilder restClientBuilder = null;
restClientBuilder = RestClient.builder(new HttpHost("公網IP", 9200, "http"));
RestHighLevelClient client = new RestHighLevelClient(restClientBuilder);
return client;
}
}
2.2.搜尋實現
用過兩款，一種是狂神的整合es高亮，一種是現在的，做的一些平均值，有點難理解，但是還是先貼出來
@Test
public void testEs() throws IOException {
SearchRequest searchRequest = new SearchRequest();
// 指定索引
searchRequest.indices("newbank");
// 1.指定檢索條件 DSL
SearchSourceBuilder builder = new SearchSourceBuilder();
// 1.1 構造檢索條件
builder.query(QueryBuilders.matchQuery("address","mill"));
// 1.2 按照年齡值聚合
TermsAggregationBuilder ageAgg = AggregationBuilders.terms("ageAgg").field("age").size(10);
builder.aggregation(ageAgg);
// 1.3 計算平均薪資
AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("balance");
builder.aggregation(balanceAvg);
System.out.println("檢索條件：" + builder.toString());
searchRequest.source(builder);
// 2.執行檢索
SearchResponse search = restHighLevelClient.search(searchRequest, GulimallElasticSearchConfig.COMMON_OPTIONS);
// 3.分析結果
// Map map = JSON.parseObject(search.toString(), Map.class);
// System.out.println(map);
// 3.1 索取所有記錄
SearchHits hits = search.getHits();
// 詳細記錄
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
// String index = hit.getIndex();
// String id = hit.getId();
String source = hit.getSourceAsString();
Account account = JSON.parseObject(source, Account.class);
System.out.println(account);
}
// 獲取分析資料
Aggregations aggregations = search.getAggregations();
// List<Aggregation> list = aggregations.asList();
// for (Aggregation aggregation : list) {
// Terms agg = aggregations.get(aggregation.getName());
// System.out.println(agg.getBuckets());
// }
Terms agg = aggregations.get("ageAgg");
for (Terms.Bucket bucket : agg.getBuckets()) {
System.out.println("年齡: " + bucket.getKeyAsString() + "-->" + bucket.getDocCount() + "人");
}
Avg avg = aggregations.get("balanceAvg");
System.out.println("平均薪資： " + avg.getValue());
}
書山有路勤為徑，學海無涯苦作舟。程式設計師不僅要懂程式碼，更要懂生活，關注我，一起進步。
