1. 程式人生 > >kafka單機不能傳送資訊和消費訊息(轉)

kafka單機不能傳送資訊和消費訊息(轉)

zookeeper和kafaka服務0.8版本以後預設是不需要配置的,但是本文遇到了一種要修改配置的情形。

 

終端A開啟zookeeper和kafaka服務後

傳送訊息

bin/kafka-console-producer.sh --broker-list localhost:9092--topic tpc

 

終端B消費資訊

 bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic tpc --from-beginning

 

但是終端A 輸入一條資訊 摁回車輸入第二條資訊的時候報錯

[2016-04-29 11:30:08,296] WARN Property topic is not valid(kafka.utils.VerifiableProperties)

message

[2016-04-29 11:30:15,574] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:15,589] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                oducer.BrokerPartitionInfo)

[2016-04-29 11:30:15,605] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:15,605] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                 oducer.BrokerPartitionInfo)

[2016-04-29 11:30:15,607] ERROR Failed to collate messagesby topic, partition due to: Failed to fetch topic metadata for topic: tpc(kafka.producer.async.Defau                ltEventHandler)

[2016-04-29 11:30:15,721] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:15,722] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                oducer.BrokerPartitionInfo)

[2016-04-29 11:30:15,730] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:15,732] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                 oducer.BrokerPartitionInfo)

[2016-04-29 11:30:15,734] ERROR Failed to collate messagesby topic, partition due to: Failed to fetch topic metadata for topic: tpc(kafka.producer.async.Defau                ltEventHandler)

[2016-04-29 11:30:15,845] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:15,847] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                 oducer.BrokerPartitionInfo)

[2016-04-29 11:30:15,856] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:15,857] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                oducer.BrokerPartitionInfo)

[2016-04-29 11:30:15,858] ERROR Failed to collate messagesby topic, partition due to: Failed to fetch topic metadata for topic: tpc(kafka.producer.async.Defau                ltEventHandler)

[2016-04-29 11:30:15,969] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:15,971] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                 oducer.BrokerPartitionInfo)

[2016-04-29 11:30:15,981] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:15,985] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                oducer.BrokerPartitionInfo)

[2016-04-29 11:30:15,985] ERROR Failed to collate messagesby topic, partition due to: Failed to fetch topic metadata for topic: tpc(kafka.producer.async.Defau                ltEventHandler)

[2016-04-29 11:30:16,097] INFO Closing socket connection to/127.0.0.1. (kafka.network.Processor)

[2016-04-29 11:30:16,099] WARN Error while fetching metadata[{TopicMetadata for topic tpc ->

No partition metadata for topic tpc due tokafka.common.LeaderNotAvailableException}] for topic [tpc]: classkafka.common.LeaderNotAvailableException (kafka.pr                 oducer.BrokerPartitionInfo)

[2016-04-29 11:30:16,103] ERROR Failed to send requests fortopics tpc with correlation ids in [0,8](kafka.producer.async.DefaultEventHandler)

[2016-04-29 11:30:16,107] ERROR Error in handling batch of 1events (kafka.producer.async.ProducerSendThread)

kafka.common.FailedToSendMessageException: Failed to sendmessages after 3 tries.

        atkafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)

        atkafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)

        atkafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)

        atkafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)

        atscala.collection.immutable.Stream.foreach(Stream.scala:594)

        atkafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)

        atkafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)

報錯很多,主要意思就是說不能建立socket ,因為要配置的這臺伺服器是有固定的ip地址的,所以推測host檔案或者 kafaka的config檔案沒有配置。

 

 

解決方案

(1)修改host檔案

vi /etc/hosts

xx.xxx.xxx.xxx  localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhostlocalhost.localdomain localhost6 localhost6.localdomain6

xx.xx.xx.xx  hub

xx.xx.xx.xx  dev1

要修改的部分是:紅色部分的xx.xx.xx.xx 這裡要變成自己伺服器的ip

 

(2)修改kafka伺服器的配置檔案

vi config/server.properties

SocketServer Settings 裡面的host.name 寫成自己的ip地址  

Zookeeper裡面 的zookeeper.connect=xx.xxx.xxx.xxx :2181

 

【原文:https://blog.csdn.net/silcon_csdn/article/details/51281774