Kafka 安裝配置

阿新 • • 發佈：2019-01-15

<span style="font-family:Microsoft YaHei;font-size:14px;">[2014-04-02 02:42:14,220] INFO Verifying properties (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,281] INFO Property broker.id is overridden to 0 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,281] INFO Property log.cleaner.enable is overridden to false (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,281] INFO Property log.dirs is overridden to /tmp/kafka-logs (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,282] INFO Property log.retention.check.interval.ms is overridden to 60000 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,282] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,282] INFO Property log.segment.bytes is overridden to 536870912 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,282] INFO Property num.io.threads is overridden to 8 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,283] INFO Property num.network.threads is overridden to 2 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,283] INFO Property num.partitions is overridden to 2 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,283] INFO Property port is overridden to 9092 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,283] INFO Property socket.receive.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,284] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,284] INFO Property socket.send.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,284] INFO Property zookeeper.connect is overridden to 192.168.40.134:2181,192.168.40.132:2181,192.168.40.133:2181 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,284] INFO Property zookeeper.connection.timeout.ms is overridden to 1000000 (kafka.utils.VerifiableProperties)
[2014-04-02 02:42:14,309] INFO [Kafka Server 0], starting (kafka.server.KafkaServer)
[2014-04-02 02:42:14,312] INFO [Kafka Server 0], Connecting to zookeeper on 192.168.40.134:2181,192.168.40.132:2181,192.168.40.133:2181 (kafka.server.KafkaServer)
[2014-04-02 02:42:14,334] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2014-04-02 02:42:14,348] INFO Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,348] INFO Client environment:host.name=ubuntu (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,348] INFO Client environment:java.version=1.6.0_37 (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,348] INFO Client environment:java.vendor=Sun Microsystems Inc. (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,348] INFO Client environment:java.home=/etc/local/jdk1.6.0_37/jre (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:java.class.path=:/usr/local/kafka_2.10-0.8.1/bin/../core/build/dependant-libs-2.8.0/*.jar:/usr/local/kafka_2.10-0.8.1/bin/../perf/build/libs//kafka-perf_2.8.0*.jar:/usr/local/kafka_2.10-0.8.1/bin/../clients/build/libs//kafka-clients*.jar:/usr/local/kafka_2.10-0.8.1/bin/../examples/build/libs//kafka-examples*.jar:/usr/local/kafka_2.10-0.8.1/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/usr/local/kafka_2.10-0.8.1/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/jopt-simple-3.2.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/kafka_2.10-0.8.1.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/log4j-1.2.15.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/metrics-annotation-2.2.0.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/metrics-core-2.2.0.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/scala-library-2.10.1.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/slf4j-api-1.7.2.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/snappy-java-1.0.5.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/zkclient-0.3.jar:/usr/local/kafka_2.10-0.8.1/bin/../libs/zookeeper-3.3.4.jar:/usr/local/kafka_2.10-0.8.1/bin/../core/build/libs/kafka_2.8.0*.jar (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:java.library.path=/etc/local/jdk1.6.0_37/jre/lib/amd64/server:/etc/local/jdk1.6.0_37/jre/lib/amd64:/etc/local/jdk1.6.0_37/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:os.version=3.8.0-29-generic (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,349] INFO Client environment:user.dir=/usr/local/kafka_2.10-0.8.1 (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,352] INFO Initiating client connection, connectString=192.168.40.134:2181,192.168.40.132:2181,192.168.40.133:2181 sessionTimeout=6000  
[email protected] (org.apache.zookeeper.ZooKeeper)
[2014-04-02 02:42:14,412] INFO Opening socket connection to server /192.168.40.134:2181 (org.apache.zookeeper.ClientCnxn)
[2014-04-02 02:42:14,779] INFO Socket connection established to ubuntu.local/192.168.40.134:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2014-04-02 02:42:14,805] INFO Session establishment complete on server ubuntu.local/192.168.40.134:2181, sessionid = 0x14521d0059e0000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2014-04-02 02:42:14,811] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2014-04-02 02:42:15,045] INFO Log directory '/tmp/kafka-logs' not found, creating it. (kafka.log.LogManager)
[2014-04-02 02:42:15,072] INFO Starting log cleanup with a period of 60000 ms. (kafka.log.LogManager)
[2014-04-02 02:42:15,077] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2014-04-02 02:42:15,170] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2014-04-02 02:42:15,176] INFO [Socket Server on Broker 0], Started (kafka.network.SocketServer)
[2014-04-02 02:42:15,310] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2014-04-02 02:42:15,387] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2014-04-02 02:42:15,527] INFO Registered broker 0 at path /brokers/ids/0 with address ubuntu:9092. (kafka.utils.ZkUtils$)
[2014-04-02 02:42:15,571] INFO [Kafka Server 0], started (kafka.server.KafkaServer)
[2014-04-02 02:42:15,654] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
</span>

建立topic：

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic mykafka

檢視Topic：

bin/kafka-topics.sh --list --zookeeper localhost:2181

顯示Topic：mykafka

檢視詳細資訊：

bin/kafktopics.sh --describe --zookeeper 192.168.40.132:2181 Topic:mykafka PartitionCount:1 ReplicationFactor:3 Configs:

Topic: mykafka Partition: 0 Leader: 133 Replicas: 133,134,132 Isr: 134傳送訊息：

bin/kafka-console-producer.sh --broker-list 192.168.40.134:9092 --topic mykafka

^C

bin/kafka-console-producer.sh --brokelist 192.168.40.134:9092 --topic mykafka

如果出現以下資訊，則需要下載slftj-nop-1.5.jar，並將其cp至kafka的libs目錄下：

<span style="font-family:Microsoft YaHei;font-size:14px;">SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. </span>

接收訊息：

bin/kafka-console-consumer.sh --zookeeper 192.168.40.133:2181 --topic mykafka --from-beginning

sfsa

asdg

sfa

asfda

sadf

三、kafka安裝配置

取數自我保護 ash cti 條件 hostname 寫入數據這也 config/server.properties broker.id 0 每一個broker在集群中的唯一表示，要求是正數。當該服務器的IP地址發生改變時，broker.id沒有變化，則不會影響

大資料入門（20）kafka安裝配置

kafka基本概念 1、kafka是一個分散式的訊息快取系統 2、kafka叢集中的伺服器都叫做broker 3、kafka有兩類客戶端，一類叫producer（訊息生產者），一類叫做consumer（訊息消費者），客戶端和broker伺服器之間採用tcp協議連線 4、kafka中不同業務系統的

windwos kafka 安裝配置

1.簡介 Kafka是一種高吞吐量的分散式釋出訂閱訊息系統。詳細介紹可查閱官網：kafka官網 2.環境搭建 2.1 安裝JDK 下載地址：jre下載 2.2 安裝Zookeeper 下載後，解壓放在目錄D:\bigdata（本文所用的目錄）下，關於zooke

Kafka安裝配置之Broker配置

Kafka安裝配置之Broker配置常規配置： broker.id

kafka安裝配置及與logstash整合

1、kafka安裝下載 wget http://mirror.bit.edu.cn/apache/kafka/0.8.2.2/kafka_2.9.1-0.8.2.2.tgz 配置zookeeper vim bin/zookeeper-server-start.

Kafka 安裝配置 windows 下

Kafka 安裝配置 windows 下標籤（空格分隔）： Kafka Kafka 核心部分需要安裝jdk, zookeeper. 安裝JDK 安裝JDK就不需要講解了，安裝完配置下JAVA_HOME，和Path,將java安裝目錄和執行檔案目錄配置下就ok拉。安裝地址: 下載安裝 zooke

ELK叢集-Kafka安裝配置(三)

安裝 kafka: tar -zxvf kafka_2.11-2.1.0.tgz mv kafka_2.11-2.1.0 /home/kafka-2.11 zookeeper: tar -zxvf zookeeper-3.4.13.tar.gz mv zookeepe

Kafka 安裝配置

<span style="font-family:Microsoft YaHei;font-size:14px;">[2014-04-02 02:42:14,220] INFO Verifying properties (kafka.utils.VerifiableProperties) [201

Kafka安裝配置

浪費了“黃金五年”的Java程式設計師，還有救嗎？ >>>

kafka manager安裝配置和使用

dht install ota manage 壓縮 etc /etc all div kafka manager安裝配置和使用 1.安裝yum源 curl https://bintray.com/sbt/rpm/rpm | sudo tee /etc/yum.rep

kafka集群安裝配置

del etc pcs zookeepe 分發 pan div 宋體 rod 1.下載安裝包 2.解壓安裝包 3.進入到kafka的config目錄修改server.properties文件進入後顯示如下: 修改log.dirs,基本上大部分都是默認配置

Kafka概述以及安裝配置

一、Kafka概述 PUBLISH & SUBSCRIBE Read and write streams of data like a messaging system. 釋出和訂閱讀取和寫入資料流，類似訊息傳遞系統。 PROCESS Write scalable stre

Elk叢集安裝+配置（Elasticsearch+Logstash+Filebeat+Kafka+Kibana）

一、部署環境 1.基礎環境：軟體版本作用 Linux Centos7.1，16g Jdk 1.8.0_151

Centons7下安裝配置Flume、Linux下安裝配置Flume、Flume的簡單使用示例、Flume整合Kafka使用

本篇文章主要介紹在windows下使用vmware虛擬機器中的Linux(Centons7)下配置安裝Flume。目前Flume 的最新版本為1.8，筆者安裝的是1.6，是Flume的一個經典版本，通常在生產環境中使用的就是這個版本，在安裝Flume前，看先它的前置條件 (1)

Window上安裝配置Zookeeper和Kafka

安裝配置Zookeeper 下載zookeeper安裝包 :http://zookeeper.apache.org/doc/r3.5.4-beta/ 解壓後可以看到：將conf下的zoo-sample.cfg更名為zoo.cfg，因為剛下下來的bin/zkEnv.cmd裡

Centons7下安裝配置Kafka、Linux下安裝配置Kafka

本篇文章主要介紹在windows下使用vmware虛擬機器中的Linux(Centons7)下配置安裝Kafka。簡要步驟： ①、下載Kafka解壓包 ②、解壓Kafka ③、修改配置檔案 ④、啟動Kafka ⑤、建立topic 詳細步驟：

mq(activemq kafka rocketmq)安裝配置筆記

ActiveMq 下載 http://activemq.apache.org/download.html 這裡下載最快 window直接用網頁下載, linux 下獲取真實路徑後用wget 命令下載 wget http://mirrors.hust.ed

Ubuntu中安裝zookeeper及kafka並配置環境變數

首先安裝zookeeper zookeeper需要jdk環境，請在jdk安裝完成的情況下安裝zookeeper 1.從官網下載zookeeper http://mirror.bit.edu.cn/apache/zookeeper/，版本可以根據自己需要選擇使用，

Kafka安裝與配置

安裝Zookeeper 安裝Kafka必須先安裝zookeeper，雖然Kafka內建了zookeeper，但是建議不用。我們全新安裝部署一套。注意：zookeeper 3.5.4版本需要JDK1.8以上版本支援 tar xzvf zookeeper-3.5.4-beta.t

kafka叢集安裝配置

1，準備三臺機器，例如IP地址分別為109.105.111.11, 109.105.111.12, 109.105.111.13的三臺機器。 2，下載kafka穩定版本，我下載的是kafka_2.10-0.9.0.0.tgz。 3，分別在三臺機器上解壓放到想安裝的目錄，我的

Kafka 安裝配置

相關推薦