1. 程式人生 > >大資料學習[11]:JAVA連線elasticsearch5.6.1操作|問題|分析

大資料學習[11]:JAVA連線elasticsearch5.6.1操作|問題|分析

摘要:通過JAVA API操作elasticsearch5.6.1的連線及連線過程中所遇到的問題,問題分析,問題解決。作為一個ES的JAVA操作的新生參考與記錄。

0. 官方文件

1. 建立工程

建立一個mave工程
這裡寫圖片描述

2. 日誌配置檔案

 建立檔名log4j2.properties,放在src的根目錄就可以,讓java可以找到;
appender.console.type = Console
appender.console.name = console
appender.console.layout.type = PatternLayout

rootLogger.level
= info rootLogger.appenderRef.console.ref = console

3. 配置pom.xml檔案

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion
>
<groupId>sparkml</groupId> <artifactId>es</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <name>es</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding
>
UTF-8</project.build.sourceEncoding> </properties> <repositories> <!-- add the elasticsearch repo --> <repository> <id>elasticsearch-releases</id> <url>https://artifacts.elastic.co/maven</url> <releases> <enabled>true</enabled> </releases> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repositories> <dependencies> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>transport</artifactId> <version>5.6.1</version> </dependency> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>x-pack-transport</artifactId> <version>5.6.1</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.9.1</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> </dependencies> </project>

4. ES中index索引資料

這裡寫圖片描述

5. demo java

public class App {
    @SuppressWarnings({ "unchecked", "resource" })
    public static void main(String[] args) {
        // on startup
        TransportClient client;
        try {
            Settings settings = Settings.builder()
                    .put("cluster.name", "esName")
                    .put("xpack.security.transport.ssl.enabled", false)
                    .put("xpack.security.user", "elastic:changeme")
                    .put("client.transport.sniff", true).build();
            client = new PreBuiltXPackTransportClient(settings)
                    .addTransportAddress(
                            new InetSocketTransportAddress(InetAddress
                                    .getByName("192.168.xxx.xxx"), 9300))
                    .addTransportAddress(
                            new InetSocketTransportAddress(InetAddress
                                    .getByName("192.168.xxx.xxx"), 9300))
                    .addTransportAddress(
                            new InetSocketTransportAddress(InetAddress
                                    .getByName("192.168.xxx.xxx"), 9300));
            GetResponse response = client.prepareGet("index", "fulltext", "2")
                    .setOperationThreaded(true).get();
            System.out.println(response);

            // on shutdown
            client.close();
        } catch (UnknownHostException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }
}

6. 執行結果

loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]
loaded plugin [org.elasticsearch.join.ParentJoinPlugin]
loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]
loaded plugin [org.elasticsearch.script.mustache.MustachePlugin]
loaded plugin [org.elasticsearch.transport.Netty3Plugin]
loaded plugin [org.elasticsearch.transport.Netty4Plugin]
loaded plugin [org.elasticsearch.xpack.XPackPlugin]
{"_index":"index","_type":"fulltext","_id":"2","_version":2,"found":true,"_source":{
    "content":"中韓漁警衝突調查:韓警平均每天扣1艘中國漁船"
}
}

7.問題

7.1 出錯1

failed to get node info for {#transport#-1}{aOvqR1DwTkSlqWAiPocgXw}{192.168.xxx.xxx}{192.168.xxx.xxx:9300}, disconnecting...
org.elasticsearch.transport.RemoteTransportException: [node][192.168.xxx.xxx:9300][cluster:monitor/nodes/liveness]
Caused by: org.elasticsearch.ElasticsearchSecurityException: missing authentication token for action [cluster:monitor/nodes/liveness]
    at org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:39) ~[?:?]
    at org.elasticsearch.xpack.security.authc.DefaultAuthenticationFailureHandler.missingToken(DefaultAuthenticationFailureHandler.java:74) ~[?:?]
    at org.elasticsearch.xpack.security.authc.AuthenticationService$AuditableTransportRequest.anonymousAccessDenied(AuthenticationService.java:553) ~[?:?]
    at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$handleNullToken$16(AuthenticationService.java:363) ~[?:?]

原因
這個因為ES安裝x-pack,所以在訪問x-pack的時候要加入使用者名稱也密碼;
在設定Settings中加入.put(“xpack.security.user”, “elastic:changeme”)。

7.2 出錯2

Exception in thread "main" java.lang.IllegalArgumentException: unknown setting [xpack.security.transport.ssl.enabled] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:293)
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:256)
    at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:139)
    at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:138)
    at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:265)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:130)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:116)
    at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:106)
    at sparkml.es.App.main(App.java:30)
    Suppressed: java.lang.IllegalArgumentException: unknown setting [xpack.security.user] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
        ... 9 more

原因:
不認識加了xpack開頭的所有引數,這個是因為缺少xpack的lib造成的。
解決:
pom.xml加上

<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>x-pack-transport</artifactId>
    <version>5.6.1</version>
</dependency>

不過一般預設的maven配置的庫對這個包是找不到,所以是要對庫進行配置一下:

<repositories>
    <!-- add the elasticsearch repo -->
    <repository>
        <id>elasticsearch-releases</id>
        <url>https://artifacts.elastic.co/maven</url>
        <releases>
            <enabled>true</enabled>
        </releases>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
</repositories>

8.題外–安裝ES5.6.1