windows下kafka源碼閱讀環境搭建
工具準備：jdk1.8,scala-2.11.11,gradle-3.1,zookeeper-3.4.5,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz
安裝jdk
安裝scala
搭建zookeeper
kafka源碼構建
解壓kafka-0.10.0.1-src.tgz，命令行進行kafka-0.10.0.1-src，執行gradle idea
註：
1)gradle idea，用國外的庫構建的下載速度很慢，在進入項目裏面，找到build.gradle文件的修改庫為阿裏鏡像庫
buildscript {
repositories {
maven{ url ‘http://maven.aliyun.com/nexus/content/groups/public/‘}
}
apply from: file(‘gradle/buildscript.gradle‘), to: buildscript
dependencies {
// For Apache Rat plugin to ignore non-Git files
classpath "org.ajoberstar:grgit:1.5.0"
classpath ‘com.github.ben-manes:gradle-versions-plugin:0.12.0‘
classpath ‘org.scoverage:gradle-scoverage:2.0.1‘
}
}
allprojects {
apply plugin: ‘idea‘
apply plugin: ‘eclipse‘
apply plugin: "jacoco"
allprojects {
repositories {
maven{ url ‘http://maven.aliyun.com/nexus/content/groups/public/‘}
}
}
2)執行gradle idea時，報錯
* What went wrong:
A problem occurred evaluating root project ‘kafka-0.10.0.1-src‘.
> Failed to apply plugin [class ‘org.gradle.api.plugins.scala.ScalaBasePlugin‘]
> No such property: useAnt for class: org.gradle.api.tasks.scala.ScalaCompileOptions
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
解決方法：打開kafka-0.10.0.1-src目錄下build.gradle文件
添加如下行
[plain] view plain copy
ScalaCompileOptions.metaClass.daemonServer = true
ScalaCompileOptions.metaClass.fork = true
ScalaCompileOptions.metaClass.useAnt = false
ScalaCompileOptions.metaClass.useCompileDaemon = false
7.用intellij idea打開kafka-0.10.0.1-src
8.復制log4j.properties文件放到core模塊的src/main/scala路徑下，這樣才能正常的顯示日誌
9.增加scala-sdk
10.配置啟動參數
11.啟動項目
12.驗證
解壓kafka_2.11-0.10.0.1.tgz文件，命令行進入bin/windows
1）創建topic
C:\webserver\kafka_2.11-0.10.0.1\bin\windows>kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --
partitions 1 --topic testMsg
Created topic "testMsg".
控制臺日誌如下：
2）執行生產者命令，產生消息：
3）執行消費者命令，消費消息：
問題：
ERROR Processor got uncaught exception. (kafka.network.Processor)
java.nio.BufferUnderflowException
解決方案：
這是客戶端的jar包和集群的版本號在小版本不一致導致，保持版本號一致即可。
scala-2.11.11,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz
本文出自 “柯浩堅” 博客，請務必保留此出處http://gauyanm.blog.51cto.com/629619/1979198
windows下kafka源碼閱讀環境搭建