1. 程式人生 > >apache分別基於三種方案實現tomcat的代理、負載均衡及會話綁定

apache分別基於三種方案實現tomcat的代理、負載均衡及會話綁定

tomcat apache

apache分別基於mod_proxy_ajp, mod_proxy_http, mod_jk三種方案實現代理、負載均衡、會話綁定及Tomcat session cluster


1、nginx, haproxy, apache(mod_proxy_ajp, mod_proxy_http, mod_jk)反代用戶請求至tomcat;

2、nginx, haproxy, apache(mod_proxy_ajp, mod_proxy_http, mod_jk)負載用戶請求至tomcat servers;額外實現session sticky;

3、tomcat session cluster;

4、tomcat session server(msm);


規劃:

172.16.1.2 apache反代服務器

172.16.1.3 tomcat A

172.16.1.10 tomcat B


說明:

如題要求,可以選擇使用nginx, haproxy, apache三個中的一個去反代用戶請求至tomcat,因為apache和Tomcat都是Apache 軟件基金會的產品,兩者有更好的兼容性,所以選擇用apache,並且會以mod_proxy_ajp, mod_proxy_http, mod_jk三種方式分別實現題目要求。


1.方案一:mod_proxy_http

vim /etc/httpd/conf.d/lb_tomcat.conf


Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED

<Proxy balancer://tcsrvs>
  BalancerMember http://172.16.1.10:8080/testapp route=TomcatA loadfactor=1
  BalancerMember http://172.16.1.3:8080/testapp route=TomcatB loadfactor=2
        ProxySet lbmethod=byrequests
        ProxySet stickysession=ROUTEID
</Proxy>

<VirtualHost *:80>
        ServerName      lb.zrs.com
        ProxyVia On
        ProxyRequests Off
        <Proxy *>
                Require all granted
        </Proxy>
        ProxyPass / balancer://tcsrvs/
        <Location />
                Require all granted
        </Location>
</VirtualHost>


分別為兩個Tomcat的server.xml配置文件中的引擎中添加jvmRoute


第一個tomcat配置為TomcatA:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatA">

第二個tomcat配置為TomcatB:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatB">


分別為兩個Tomcat Server的testapp提供測試頁面:

tomcatA:

# mkdir -pv /usr/share/tomcat/webapps/testapp/{WEB-INF,classes,lib}

# vim /usr/share/tomcat/webapps/testapp/index.jsp

添加如下內容:


<%@ page language="java" %>
<html>
  <head><title>TomcatA</title></head>
  <body>
    <h1><font color="red">TomcatA.zrs.com</font></h1>
    <table align="centre" border="1">
      <tr>
        <td>Session ID</td>
    <% session.setAttribute("zrs.com","zrs.com"); %>
        <td><%= session.getId() %></td>
      </tr>
      <tr>
        <td>Created on</td>
        <td><%= session.getCreationTime() %></td>
     </tr>
    </table>
  </body>
</html>


tomcatB:

# mkdir -pv /usr/share/tomcat/webapps/testapp/{WEB-INF,classes,lib}

# vim /usr/share/tomcat/webapps/testapp/index.jsp

添加如下內容:


<%@ page language="java" %>
<html>
  <head><title>TomcatB</title></head>
  <body>
    <h1><font color="blue">TomcatB.zrs.com</font></h1>
    <table align="centre" border="1">
      <tr>
        <td>Session ID</td>
    <% session.setAttribute("zrs.com","zrs.com"); %>
        <td><%= session.getId() %></td>
      </tr>
      <tr>
        <td>Created on</td>
        <td><%= session.getCreationTime() %></td>
     </tr>
    </table>
  </body>
</html>


客戶端測試,因為做了session sticky,會一直調度到後端一臺服務器上,如我的一直是TomcatA

技術分享圖片

關掉TomcatA,便會調度到TomcatB上。

技術分享圖片


2.方案二:mod_proxy_ajp

使用mod_proxy_ajp,僅需要把mod_proxy_http的http改為ajp,端口8080改為8009即可,需要註意的是前端用了ajp,可以把後端的tomcat的8080端口註釋掉,不讓其監聽,從而使安全性更高。


vim /etc/httpd/conf.d/lb_tomcat.conf


Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED

<Proxy balancer://tcsrvs>
  BalancerMember ajp://172.16.1.10:8009/testapp route=TomcatA loadfactor=1
  BalancerMember ajp://172.16.1.3:8009/testapp route=TomcatB loadfactor=2
        ProxySet lbmethod=byrequests
        ProxySet stickysession=ROUTEID
</Proxy>

<VirtualHost *:80>
        ServerName      lb.qhdlink.com
        ProxyVia On
        ProxyRequests Off
        <Proxy *>
                Require all granted
        </Proxy>
        ProxyPass / balancer://tcsrvs/
        <Location />
                Require all granted
        </Location>
</VirtualHost>

關閉session sticky,客戶端查看

技術分享圖片

技術分享圖片

3.方案三:mod_jk


此方案需要mod_jk模塊

需要安裝這個文件tomcat-connectors-VERSION.tar.gz


配置文件:

vim /etc/httpd/conf.d/mod_jk_lb.conf

LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd/conf.d/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* tcsrvs
JkMount /jk-status Stat

vim /etc/httpd/conf.d/workers.properties

worker.list=tcsrvs,Stat
worker.TomcatA.host=172.16.1.10
worker.TomcatA.port=8009
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.TomcatB.host=172.16.1.3
worker.TomcatB.port=8009
worker.TomcatB.type=ajp13
worker.TomcatB.lbfactor=2
worker.tcsrvs.type=lb
worker.tcsrvs.balance_workers=TomcatA,TomcatB
worker.tcsrvs.sticky_session=1
worker.Stat.type=status


4.tomcat session cluster


在兩個tomcat的server.xml配置啟用集群

添加如下配置內容,並且註釋掉8080端口

需要註意的是下面的內容中,將address="auto"的auto改為該主機的IP地址


<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"

channelSendOptions="8">

<Manager className="org.apache.catalina.ha.session.DeltaManager"

expireSessionsOnShutdown="false"

notifyListenersOnReplication="true"/>


<Channel className="org.apache.catalina.tribes.group.GroupChannel">

<Membership className="org.apache.catalina.tribes.membership.McastService"

address="228.100.0.4"

port="45564"

frequency="500"

dropTime="3000"/>

<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"

address="auto"

port="4000"

autoBind="100"

selectorTimeout="5000"

maxThreads="6"/>


<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">

<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>

</Sender>


<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>

<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>

</Channel>


<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"

filter=""/>

<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>


<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"

tempDir="/tmp/war-temp/"

deployDir="/tmp/war-deploy/"

watchDir="/tmp/war-listen/"

watchEnabled="false"/>

<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>

<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>

</Cluster>



並且在Host中添加一個Context組件

<Host name="localhost" appBase="webapps"

unpackWARs="true" autoDeploy="true">

<Context path="/test" docBase="/myapps/webapps" reloadable="true"/>


創建目錄

[root@zrs tomcat]# mkdir -pv /myapps/webapps/{classes,lib,WEB-INF}


創建測試頁面

跟上面的一樣,創建AB兩個不同的頁面

[root@zrs tomcat]# vim /myapps/webapps/index.jsp


復制web.xml到指定目錄

[root@zrs tomcat]# cp web.xml /myapps/webapps/WEB-INF/


編輯WEB-INF/web.xml

在任意非段落外添加<distributable/>

[root@zrs tomcat]# vim /myapps/webapps/WEB-INF/web.xml


反代服務器配置

[root@director conf.d]# vim mod_jk_proxy.conf


LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd/conf.d/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* tcsrvs
JkMount /jk-status Stat



[root@director conf.d]# vim workers.properties


worker.list=tcsrvs,Stat
worker.TomcatA.host=172.16.1.10
worker.TomcatA.port=8009
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.TomcatB.host=172.16.1.3
worker.TomcatB.port=8009
worker.TomcatB.type=ajp13
worker.TomcatB.lbfactor=2
worker.tcsrvs.type=lb
worker.tcsrvs.balance_workers=TomcatA,TomcatB
worker.tcsrvs.sticky_session=0
worker.Stat.type=status

分別開啟三臺主機的httpd服務和tomcat服務


客戶端測試

可以看到反代服務器以ABB的次序輪詢後端兩臺tomcat服務器,但是ID不變,從而實現session cluster。

技術分享圖片

技術分享圖片

5.tomcat session server(msm)

接著剛才的配置環境,這個構建方案的原理是將所有會話放在同一臺後端tomcat服務器上,當後端這臺tomcat服務器宕機時,整個搭建便無法正常提供服務,所以也要給tomcat服務器做高可用。


這個構建方案需要memcached作為旁掛式緩存服務器,所以需要提前下載並啟動memcached服務。


停止兩臺tomcat服務,還原server.xml配置文件

註釋掉8080端口,在Engine中添加jvmRoute,如下


<Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatA" >

<Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatB" >


在Host段添加配置內容


<Context path="/test" docBase="/myapps/webapps" reloadable="true">

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"

memcachedNodes="n1:172.16.1.10:11211,n2:172.16.1.3:11211"

failoverNodes="n1"

requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js|html|htm)$"

transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"

/>

</Context>


導入類文件

memcached-session-manager項目地址,http://code.google.com/p/memcached-session-manager/


下載如下jar文件至各tomcat節點的tomcat安裝目錄下的lib目錄中,其中的${version}要換成你所需要的版本號,tc${6,7,8}要換成與tomcat版本相同的版本號。

memcached-session-manager-${version}.jar

memcached-session-manager-tc${6,7,8}-${version}.jar

spymemcached-${version}.jar

msm-javolution-serializer-${version}.jar

javolution-${version}.jar


將所有的類放在/usr/share/java/tomcat目錄中。


啟動tomcat服務即可看到反代服務器以AB的次序輪詢後端兩臺tomcat服務器,但是session ID不變。


apache分別基於三種方案實現tomcat的代理、負載均衡及會話綁定