1. 程式人生 > >Kafka入門,producer和consumer與hive

Kafka入門,producer和consumer與hive

{

       "name":"hdfs-hive-sink-03",

       "config":{

              "connector.class":"io.confluent.connect.hdfs.HdfsSinkConnector",

              "tasks.max":"1",

              "topics":"text_source3",

              "hdfs.url":"hdfs://localhost:9000",

              "flush.size":"1",

              "hive.integration":"true",

              "hive.metastore.uris":"thrift://localhost:9083",

              "schema.compatibility":"BACKWARD"

       }

}

kafka-avro-console-producer --broker-list localhost:9092 --topic text_source3 --property vae.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}'

{"f1":"f1_001"}

輸入(生產者)

輸出(消費者)

kafka-topics --delete --zookeeper localhost:2181 --topic test_source

刪除topic

kafka-topics --zookeeper localhost:2181 --list

查詢topic