.NET 分散式搜尋 elasticsearch.net NEST

阿新 • • 發佈：2019-01-19

我們公司（電商）目前使用 elasticsearch。對於一個3億資料量索引查詢速度都是在毫秒級別，不過實現都是基於Java的client. 本人因為是.NET出身，雖然現在做JAVA。但是對.NET情有獨鍾。所以閒暇時間研究了下 elasticsearch的.NET 的CLIENT的NEST.希望.NET人員多使用 elasticsearch 來做搜尋服務解決.NET一直沒有好的搜尋服務。本人以前也研究過 Lucene.NET。但是無法實現叢集及分散式。

本人也參考了很多前輩部落格和和本公司做搜尋的同事和 google。走了很多彎路。所以希望後來人別走很多彎路。

Elasticsearch一般至少最少3臺服務。2臺伺服器時候1一臺是主分片。另外一臺永遠都是副本所以在服務，無法實現叢集式搜尋。只能實現主從備份模式。

所以伺服器建議3臺以上。這樣分片才會平衡到3臺伺服器上。

參考資料

安裝Elasticsearch

網上已經有很多資料了。所以這裡不再寫了

1、VS2012安裝NEST

PM> Install-Package NEST -version 1.7.2

我安裝的是1.7.2的客戶端版本。

2、建立連線

 public  static ElasticClient GetElasticClient(string indexName)
        {
            var connectString = "http://127.0.0.1:9200";
            var 
 nodesStr = connectString.Split('|');
            var nodes = nodesStr.Select(s => new Uri(s)).ToList();
            var connectionPool = new SniffingConnectionPool(nodes);
            var settings = new ConnectionSettings(connectionPool);
            if (!string.IsNullOrWhiteSpace(indexName))
            {
                settings.SetDefaultIndex(indexName);
            }
            var 
 client = new ElasticClient(settings);
            return client;

        }

3、建立索引

建立索引只需要呼叫一次即可。

 索引結構如下：

[ElasticType(IdProperty = "id", Name = "order")]
public class Order
{
 [ElasticProperty(Name = "id", Type = FieldType.Long, Index = FieldIndexOption.NotAnalyzed)]
  public long Id { get; set; }


public string Content { get; set; }
[ElasticProperty(Name = "firstname", Type = FieldType.String, Index = FieldIndexOption.Analyzed)]
public string Firstname { get; set; }

[ElasticProperty(Name = "lastname", Type = FieldType.String, Index = FieldIndexOption.NotAnalyzed)]
public string Lastname { get; set; }


 [ElasticProperty(Name = "totalprice", Type = FieldType.Double, Index = FieldIndexOption.NotAnalyzed)]
public Double TotalPrice { get; set; }

[ElasticProperty(Name = "createtime", Type = FieldType.Date, Index = FieldIndexOption.Analyzed)]
public DateTime Createtime { get; set; }

    }

[ElasticType(IdProperty = “id”, Name = “order”)]

這個IdProperty欄位屬性是搜尋在做分片時候需要HASH物件。所以選擇這個可以根據資料庫的分割槽鍵或者資料庫主鍵來選擇具體哪個欄位來做IdProperty。一般都是選擇資料庫主鍵

Content：這個欄位未定義屬性。所以這個欄位不能拿來做查詢條件。

要做查詢條件的欄位都必須加上

[ElasticProperty(Name = “lastname”, Type = FieldType.String, Index = FieldIndexOption.NotAnalyzed)]
或者

[ElasticProperty(Name = “createtime”, Type = FieldType.Date, Index = FieldIndexOption.Analyzed)]

這兩個的區別是
NotAnalyzed 是不分詞只做索引欄位

Analyzed 作為查詢欄位並且分詞

所以建立索引必須規劃好每個欄位是否分詞及是否查詢。

      /// <summary>
        /// 建立索引
        /// </summary>
        /// <param name="indexName"></param>
        private static void CreateIndex(string indexName)
        {

            var client = ElasticsearchHelper.GetElasticClient("");
            //client 不能給 default
            client.CreateIndex(r => r.Index(indexName).NumberOfReplicas(1).NumberOfShards(20));
            client.Map<Order>(m => m.MapFromAttributes()); 
        }

這裡是建立ORDER的索引並且每個分片一個副本總共20個分片。

20分片實際要根據業務資料量多少來決定。

4、刪除索引

      /// <summary>
        /// 刪除索引
        /// </summary>
        private static void DeleteIndex()
        {
            var client = ElasticsearchHelper.GetElasticClient("order");
            var result = client.DeleteIndex(r => r.Index("order"));
            System.Console.WriteLine(result.IsValid);
        }

5、寫入資料

    private static  void WriteIndex()
        {
            var client = ElasticsearchHelper.GetElasticClient("order");
            long i = GetMax();
            while (true)
            {
                i++;
                SetMax(i);
                Order order = new Order()
                {
                    Id = i,
                    Firstname = "我愛天安門",
                    Lastname = "外派",
                    TotalPrice =Convert.ToDouble(i),
                    Content = "天使3412341234" + i.ToString(),
                    Createtime = DateTime.Now
                };
                var addResult = client.Index(order);
                System.Console.WriteLine("第" + i + "行記錄" + addResult.Created);

            }

        }

        static long GetMax()
        {
            long reslt = 0;
            try
            {
                string k = System.IO.File.ReadAllText("1.txt");
                reslt = Convert.ToInt64(k);
            }
            catch (Exception)
            {

            }
            return reslt;
        }

        static long SetMax(long max)
        {
            long reslt = 0;
            try
            {
                System.IO.File.WriteAllText("1.txt", max.ToString());
            }
            catch (Exception)
            {

            }
            return reslt;
        }

6、查詢

#region 求和
var client = ElasticsearchHelper.GetElasticClient("order");
QueryContainer termQuery = new TermQuery() { Field = "lastname", Value = "求和" };
var result = client.Search<Order>(s => s
                .Aggregations(a => a
                    .Sum("my_sum_agg", sa => sa
                        .Field(p => p.TotalPrice)
                    )
                )
            );
var agg = result.Aggs.Sum("my_sum_agg");


var searchResults = client.Search<Order>(s => s
              //  .Query(termQuery)  //帶篩選條件
                .Aggregations(r => r.Terms("firstname", r1 => r1.Field(r2 => r2.Lastname)
                    .OrderAscending("sumprice")
                    .Aggregations(y => y.Sum("sumprice", y1 => y1.Field(y2 => y2.TotalPrice))))));

var carTypes = searchResults.Aggs.Terms("firstname");
            List<double> re=new List<double>();
            foreach (var carType in carTypes.Items)
            {
                string key = carType.Key;
                System.Console.WriteLine("key:" + key + " total:" + carType.Sum("sumprice").Value);
                re.Add((double)carType.Sum("sumprice").Value);
            }
            //List<SumTotalPrice> orders = searchResults.Documents.ToList();
         //   System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total);
            //System.Console.WriteLine(searchResults.RequestInformation);
            System.Console.ReadLine();

            #endregion


  #region 多條件精確匹配
            var client = ElasticsearchHelper.GetElasticClient("order");
            QueryContainer termQuery = new TermQuery() { Field = "firstname", Value = "測試" };
            QueryContainer wholeWordQuery = new TermQuery() { Field = "lastname", Value = "addddd" };
            //非索引欄位。無法查詢 此查詢條件無效
            QueryContainer termQuery1 = new TermQuery() { Field = "content", Value = "  天空是百度的1696" };
            termQuery = (termQuery || wholeWordQuery) || termQuery1;

            var searchResults = client.Search<Order>(s => s
                .From(0)
                .Size(10)
                .Query(termQuery)
                );
            List<Order> orders = searchResults.Documents.ToList();
            System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total);
            System.Console.ReadLine();

            #endregion

               #region 精確匹配
            var client = ElasticsearchHelper.GetElasticClient("order");
            var searchResults = client.Search<Order>(s => s
              .From(0)
              .Size(10)
              .Query(q => q.Term(r => r.OnField(k => k.Firstname).Value("棉花")))
              .Highlight(h => h.OnFields(e => e.OnField("firstname")
                                .PreTags("<b style='color:black'>")
                                .PostTags("</b>")))
              .Sort(r => r.Descending().OnField(q => q.Createtime))
              );
            List<Order> orders = searchResults.Documents.ToList();

            System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total);
            System.Console.ReadLine();

            #endregion

          //  //查詢索引欄位 帶 褲的  類似 資料庫的  like '褲%'
            var client = ElasticsearchHelper.GetElasticClient("order");
            var searchResults = client.Search<Order>(s => s
          .From(0)
          .Size(10)
          .QueryString("褲*")
          );
            System.Console.WriteLine(" total:" + searchResults.Total);
            System.Console.ReadLine();


            #region ID查詢

            var searchResults = client.Search<Order>(s => s
               .From(0)
               .Size(10)
               .Query(q => q.Ids(r=>r.Values(19)))
               );

            #endregion

 #region 時間範圍查詢
            var client = ElasticsearchHelper.GetElasticClient("order");
          //  QueryContainer termQuery = new TermQuery() { Field = "firstname", Value = "測試" };
            var format = "yyyy-MM-dd'T'HH:mm:ss.fff+08:00";
            var lowerBound = DateTime.Now.AddMinutes(-20);

            var searchResults = client.Search<Order>(s => s
               // .Query(r => termQuery && r.Range(st => st.OnField(st1 => st1.Createtime).GreaterOrEquals(lowerBound, format)))
                .Query(r => r.Range(st => st.OnField(st1 => st1.Createtime).GreaterOrEquals(lowerBound, format)))
                .SortDescending(r=>r.Createtime)
                .From(0)
                .Size(10)
                );
            List<Order> orders = searchResults.Documents.ToList();
             var k=  searchResults.RequestInformation;
            System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total);
            System.Console.ReadLine();

            #endregion

7、刪除資料

     private static  void DeleteRowById(long id)
        {
            var client = ElasticsearchHelper.GetElasticClient("order");

            client.Delete<Order>(r => r.Id(id));

        }

轉載自：http://blog.csdn.net/kingson88/article/details/51252606

.NET 分散式搜尋 elasticsearch.net NEST

.NET 分散式搜尋 elasticsearch.net NEST

分散式搜尋Elasticsearch——刪除指定索引

分散式搜尋elasticsearch java API 之 highlighting （對搜尋結果的高亮顯示）

分散式搜尋elasticsearch 基本概念

分散式搜尋elasticsearch java API 之（六）------批量新增刪除索引

分散式搜尋elasticsearch配置檔案elasticsearch.yml詳解

分散式搜尋Elasticsearch——QueryBuilders.matchPhrasePrefixQuery

分散式搜尋elasticsearch java API 之（八）------使用More like this實現基於內容的推薦

分散式搜尋Elasticsearch——QueryBuilders.matchPhraseQuery

分散式搜尋elasticsearch 搜尋結果排序不一致性問題

ASP.NET Web API + Elasticsearch 6.x 快速做個全文搜尋

Elasticsearch.Net Nest 5.0.0　用法

elasticsearch.net search入門使用指南中文版(翻譯)

Fire Net —— dfs搜尋

ASP.NET Core使用Elasticsearch記錄NLog日誌

Elasticsearch 6.5.3 釋出，分散式搜尋和資料分析引擎

Elasticsearch.net專案實戰

大資料篇：Elasticsearch分散式搜尋與分析引擎

Elasticsearch.Net 官網示例的坑

ASP.NET Web API + Elasticsearch 6.x 快速做個全文搜索

.NET 分散式 搜尋 elasticsearch.net NEST

相關推薦

.NET 分散式搜尋 elasticsearch.net NEST