Storm-0.10.0叢集安裝
阿新 • • 發佈：2018-11-05
一、環境
兩臺虛擬機器，ubuntu-14.04.3
二、關閉防火牆，配置hosts
[email protected]:~# cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 ubuntu
192.168.254.130 storm1
192.168.254.131 storm2三、安裝Java（JDK 6+），並配置環境變數
[email protected]:~# cat /etc/profile export JAVA_HOME=/usr/local/jdk1.7.0_80 export JRE_HOME=/$JAVA_HOME/jre export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
四、安裝Python（2.6.6+）
確定系統自帶的Python版本，如果是2.6.6+可以不用再安裝。
[email protected]:~# python -V
Python 2.7.6[email protected]:/usr/local# wget http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.7/zookeeper-3.4.7.tar.gz[email protected]:/usr/local# tar -zxvf zookeeper-3.4.7.tar.gz[email protected]:/usr/local# cd zookeeper-3.4.7/conf [email protected]:/usr/local/zookeeper-3.4.7/conf# cp -p zoo_sample.cfg zoo.cfg [email protected]:/usr/local/zookeeper-3.4.7/conf# vim zoo.cfg # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/opt/zookeeper/data # the port at which the clients will connect clientPort=2181 server.1=storm1:7000:7001 server.2=storm2:7000:7001 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1
# 手動建立dataDir目錄
[email protected]:/opt# mkdir zookeeper
[email protected]:/opt# cd zookeeper
[email protected]:/opt/zookeeper# mkdir data# 在dataDir目錄下建立myid檔案，寫id號，用來標識當前主機
[email protected]:/opt/zookeeper/data# echo "1" > myid# 啟動ZooKeeper
[email protected]:/usr/local/zookeeper-3.4.7# bin/zkServer.sh start
[email protected]:/usr/local/zookeeper-3.4.7# bin/zkServer.sh start
# 檢視ZooKeeper狀態
[email protected]:/usr/local/zookeeper-3.4.7# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.7/bin/../conf/zoo.cfg
Mode: follower
[email protected]:/usr/local/zookeeper-3.4.7# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.7/bin/../conf/zoo.cfg
Mode: leader# 下載安裝Storm
[email protected]:/usr/local# wget wget http://mirrors.cnnic.cn/apache/storm/apache-storm-0.10.0/apache-storm-0.10.0.tar.gz
[email protected]:/usr/local# tar -zxvf apache-storm-0.10.0.tar.gz
[email protected]:/usr/local# mv apache-storm-0.10.0 storm-0.10.0# 省略...
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "storm1"
- "storm2"
#
nimbus.host: "storm1"
storm.local.dir: "/opt/storm/data"
supervisor.slots.port:
- 6700
- 6701
- 6702
- 6703
nimbus.childopts: "-Xmx1024m"
ui.childopts: "-Xmx768m"
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "server1"
# - "server2"
## Metrics Consumers
# topology.metrics.consumer.register:
# - class: "backtype.storm.metric.LoggingMetricsConsumer"
# parallelism.hint: 1
# - class: "org.mycompany.MyMetricsConsumer"
# parallelism.hint: 1
# argument:
# - endpoint: "metrics-collector.mycompany.org"# 建立storm.local.dir目錄
[email protected]:/opt# mkdir storm
[email protected]:/opt/storm# mkdir data
[email protected]:/opt/storm/data# pwd
/opt/storm/data# 啟動Storm（確保ZooKeeper已正常啟動）
[email protected]:/usr/local/storm-0.10.0# bin/storm nimbus >/dev/null 2>&1 &
[email protected]:/usr/local/storm-0.10.0# bin/storm supervisor >/dev/null 2>&1 &
[email protected]:/usr/local/storm-0.10.0# bin/storm ui >/dev/null 2>&1 &# 檢視啟動程序
[email protected]:~# jps
2658 QuorumPeerMain
2696 nimbus
2813 core
3334 Jps
[email protected]:~# jps
2673 supervisor
3287 Jps
2632 QuorumPeerMain
七、遇到的問題
如果出現IPv6引起的連線問題可以修改Storm啟動JVM引數，如下：
nimbus.childopts: "-Xmx1024m -Djava.net.preferIPv4Stack=true"
ui.childopts: "-Xmx768m -Djava.net.preferIPv4Stack=true"