目錄：

1、需求

2、參考

3、資料和配置

4、展現

5、注意事項

————————————————————————————–

1、需求

參考官網，使用Linux向Druid傳送資料和查詢資料

2、參考

配置來源 : 使用druid的預設配置檔案
/home/druid/druid-0.9.2/quickstart/wikiticker-index.json

3、資料和配置

1、將官網的資料中的日期修改為當前日期，我只修改了YYYY-MM-DD
2、將basicdata.json放入到HDFS，路徑為：/user/druid/basicdata.json
3、將預設配置的dimensions列名，修改為basicdata.json的列名

測試的資料 ：basicdata.json

{"timestamp": "2017-03-17T01:02:33Z", "page": "Gypsy Danger", "language" : "en", "user" : "nuclear", "unpatrolled" : "true", "newPage" : "true", "robot": "false", "anonymous": "false", "namespace":"article", "continent":"North America", "country":"United States", "region":"Bay Area"
 
, "city":"San Francisco", "added": 57, "deleted": 200, "delta": -143}
{"timestamp": "2017-03-17T03:32:45Z", "page": "Striker Eureka", "language" : "en", "user" : "speed", "unpatrolled" : "false", "newPage" : "true", "robot": "true", "anonymous": "false", "namespace":"wikipedia", "continent":"Australia", "country
 
":"Australia", "region":"Cantebury", "city":"Syndey", "added": 459, "deleted": 129, "delta": 330}
{"timestamp": "2017-03-17T07:11:21Z", "page": "Cherno Alpha", "language" : "ru", "user" : "masterYi", "unpatrolled" : "false", "newPage" : "true", "robot": "true", "anonymous": "false", "namespace":"article", "continent":"Asia", "country":"Russia", "region":"Oblast", "city":"Moscow", "added": 123, "deleted": 12, "delta": 111}
{"timestamp": "2017-03-17T11:58:39Z", "page": "Crimson Typhoon", "language" : "zh", "user" : "triplets", "unpatrolled" : "true", "newPage" : "false", "robot": "true", "anonymous": "false", "namespace":"wikipedia", "continent":"Asia", "country":"China", "region":"Shanxi", "city":"Taiyuan", "added": 905, "deleted": 5, "delta": 900}
{"timestamp": "2017-03-17T12:41:27Z", "page": "Coyote Tango", "language" : "ja", "user" : "cancer", "unpatrolled" : "true", "newPage" : "false", "robot": "true", "anonymous": "false", "namespace":"wikipedia", "continent":"Asia", "country":"Japan", "region":"Kanto", "city":"Tokyo", "added": 1, "deleted": 10, "delta": -9}

測試的配置： data_schema.json

{
  "type" : "index_hadoop",
  "spec" : {
    "ioConfig" : {
      "type" : "hadoop",
      "inputSpec" : {
        "type" : "static",
        "paths" : "/user/druid/basicdata.json"
      }
    },
    "dataSchema" : {
      "dataSource" : "silentwolf",
      "granularitySpec" : {
        "type" : "arbitrary",
        "segmentGranularity" : "day",
        "queryGranularity" : "none",
        "intervals" : ["2017-03-17/2017-03-18"]
      },
      "parser" : {
        "type" : "hadoopyString",
        "parseSpec" : {
          "format" : "json",
          "dimensionsSpec" : {
            "dimensions" : [
        "page",
        "language",
        "user",
        "unpatrolled",
        "newPage",
        "robot",
        "anonymous",
        "namespace",
        "continent",
        "country",
        "region",
        "city"
         ]
          },
          "timestampSpec" : {
            "format" : "auto",
            "column" : "timestamp"
          }
        }
      },
      "metricsSpec" : [
        {
          "name" : "count",
          "type" : "count"
        },
        {
          "name" : "added",
          "type" : "longSum",
          "fieldName" : "added"
        },
        {
          "name" : "deleted",
          "type" : "longSum",
          "fieldName" : "deleted"
        },
        {
          "name" : "delta",
          "type" : "longSum",
          "fieldName" : "delta"
        }
      ]
    },
    "tuningConfig" : {
      "type" : "hadoop",
      "jobProperties" : {}
    }
  }
}

測試的查詢配置：queryall.json

    {
        "queryType": "timeseries",
        "dataSource": "silentwolf",
        "intervals": [ "2017-03-17/2017-03-18" ],
        "granularity": "day",
        "aggregations": [
            {"type": "count", "name": "count"},
            { "name" : "deleted","type" : "longSum", "fieldName" : "deleted"},
            { "name" : "delta","type" : "longSum","fieldName" : "delta"}
     ]

    }

4、展現

傳送命令

[root@tagtic-master boke]# curl -X 'POST' -H 'Content-Type: application/json' -d @data_schema.json tagtic-master:18090/druid/indexer/v1/task

查詢命令

[root@tagtic-master boke]# curl -X POST 'tagtic-slave01:18082/druid/v2/?pretty' -H 'Content-Type:application/json' -d @queryall.json 

傳送、查詢、資料展現

資料傳送狀態

5、注意事項

1、找到Druid叢集中broker的server和埠，我的broker的埠為18082

[[email protected] yuhui]# ps -ef | grep broker
druid     52680  52675  1 2月20 ?       06:31:04 java -server -Xms16g -Xmx16g -XX:MaxDirectMemorySize=4096m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djava.io.tmpdir=var/tmp -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -cp conf/druid/_common:conf/druid/broker:lib/* io.druid.cli.Main server broker
root      89216  67823  0 17:03 pts/0    00:00:00 grep --color=auto broker

2、測試資料要放到HDFS上面

3、dimensions中的列名不要和metricsSpec中的name一樣

       如果您喜歡我寫的博文，讀後覺得收穫很大，不妨小額贊助我一下，讓我有動力繼續寫出高質量的博文，感謝您的讚賞！！！