1. 程式人生 > >遠端連結Kafka總是那麼個錯---Failed to send messages after 3 tries

遠端連結Kafka總是那麼個錯---Failed to send messages after 3 tries

如果是在同一臺機器(localhost),Kafka和Java Client工作正常,Producer和Consumer都能正常傳送和接收訊息,但是一旦部署到兩臺機器,則預設配置的話不能正常工作。會出現“kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries”的錯誤。

[2015-07-02 15:15:39,295] WARN Error while fetching metadata [{TopicMetadata for topic datadog-dev -> 
No partition metadata for
topic datadog-dev due to kafka.common.LeaderNotAvailableException}] for topic [datadog-dev]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo) [2015-07-02 15:15:39,295] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic:
datadog-dev (kafka.producer.async.DefaultEventHandler) [2015-07-02 15:15:39,399] WARN Error while fetching metadata [{TopicMetadata for topic datadog-dev -> No partition metadata for topic datadog-dev due to kafka.common.LeaderNotAvailableException}] for topic [datadog-dev]: class kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo)
[2015-07-02 15:15:39,399] ERROR Failed to send requests for topics datadog-dev with correlation ids in [9,16] (kafka.producer.async.DefaultEventHandler) [2015-07-02 15:15:39,399] ERROR Error in handling batch of 4 events (kafka.producer.async.ProducerSendThread) kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90) at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67) at scala.collection.immutable.Stream.foreach(Stream.scala:547) at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66) at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)

解決方法其實很簡單,只需要在Kafka的配置檔案server.properties中,設定好主機名即可:

# Hostname the broker will bind to. If not set, the server will bind to all interfaceshost.name=queue-server1

究其原因,其實從註釋裡,我們可以知道,這是一個指定broker的地址(嚴格來說是所監聽的網路介面,或者網絡卡),同時,也可以看出它還和下面的屬性相關。

# Hostname the broker will advertise to producers and consumers. If not set, it uses the

# value for “host.name” if configured. Otherwise, it will use the value returned from

# java.net.InetAddress.getCanonicalHostName().

#advertised.host.name=

也就是說,producer和consumer是通過這個主機名(advertised.host.name)來連線broker的,而如果這個值沒有設定,則會使用上面的host.name的值,如果上面的host.name也沒有設定,則會使用java.net.InetAddress.getCanonicalHostName()獲取的值。

從Zookeeper中可以看出,預設的時候該broker是localhost,從其他機器訪問當然不可能成功。