Hadoop生態圈-Flume的組件之自定義攔截器(interceptor)
阿新 • • 發佈:2018-06-21
events nbsp sin capacity figure IV mem nap code
Hadoop生態圈-Flume的組件之自定義攔截器(interceptor)
作者:尹正傑
版權聲明:原創作品,謝絕轉載!否則將追究法律責任。
本篇博客只是舉例了一個自定義攔截器的方法,測試字節傳輸速度。
1>.自定義interceptor方法
2>.打包並將其發送到 /soft/flume/lib下
[yinzhengjie@s101 ~]$ cd /soft/flume/lib/ [yinzhengjie@s101 lib]$ [yinzhengjie@s101 lib]$ ll| grep MyFlume -rw-r--r-- 1 yinzhengjie yinzhengjie 5231 Jun 20 18:53 MyFlume-1.0-SNAPSHOT.jar [yinzhengjie@s101 lib]$ [yinzhengjie@s101 lib]$ rm -rf MyFlume-1.0-SNAPSHOT.jar [yinzhengjie@s101 lib]$ [yinzhengjie@s101 lib]$ rz [yinzhengjie@s101 lib]$ [yinzhengjie@s101 lib]$ ll | grep MyFlume -rw-r--r-- 1 yinzhengjie yinzhengjie 8667 Jun 20 21:02 MyFlume-1.0-SNAPSHOT.jar [yinzhengjie@s101 lib]$ [yinzhengjie@s101 lib]$
3>.編寫agent的配置文件
[yinzhengjie@s101 ~]$ more /soft/flume/conf/yinzhengjie_myInterceptor.conf # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 8888 # Describe the sink a1.sinks.k1.type= cn.org.yinzhengjie.interceptor.MyInterceptor a1.sinks.k1.speed = 1 # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 100000 a1.channels.c1.transactionCapacity = 10000 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1 [yinzhengjie@s101 ~]$ [yinzhengjie@s101 ~]$
4>.啟動flume並測試
Hadoop生態圈-Flume的組件之自定義攔截器(interceptor)