1. 程式人生 > >sharding-jdbc結合mybatis實現分庫分表功能

sharding-jdbc結合mybatis實現分庫分表功能

源自:https://www.cnblogs.com/zwt1990/p/6762135.html

最近忙於專案已經好久幾天沒寫部落格了,前2篇文章我給大家介紹了搭建基礎springMvc+mybatis的maven工程,這個簡單框架已經可以對付一般的小型專案。但是我們實際專案中會碰到很多複雜的場景,比如資料量很大的情況下如何保證效能。今天我就給大家介紹資料庫分庫分表的優化,本文介紹mybatis結合噹噹網的sharding-jdbc分庫分表技術(原理這裡不做介紹)

  首先在pom檔案中引入需要的依賴

<dependency>
            <groupId>com.dangdang</
groupId> <artifactId>sharding-jdbc-core</artifactId> <version>1.4.2</version> </dependency> <dependency> <groupId>com.dangdang</groupId> <artifactId>sharding-jdbc-config-spring</artifactId
> <version>1.4.0</version> </dependency>

  二、新建一個sharding-jdbc.xml檔案,實現分庫分表的配置

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
xmlns:tx="http://www.springframework.org/schema/tx" xmlns:rdb="http://www.dangdang.com/schema/ddframe/rdb" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.dangdang.com/schema/ddframe/rdb http://www.dangdang.com/schema/ddframe/rdb/rdb.xsd"> <rdb:strategy id="tableShardingStrategy" sharding-columns="user_id" algorithm-class="com.meiren.member.common.sharding.MemberSingleKeyTableShardingAlgorithm"/> <rdb:data-source id="shardingDataSource"> <rdb:sharding-rule data-sources="dataSource"> <rdb:table-rules> <rdb:table-rule logic-table="member_index" actual-tables="member_index_tbl_${[0,1,2,3,4,5,6,7,8,9]}${0..9}" table-strategy="tableShardingStrategy"/> <rdb:table-rule logic-table="member_details" actual-tables="member_details_tbl_${[0,1,2,3,4,5,6,7,8,9]}${0..9}" table-strategy="tableShardingStrategy"/> </rdb:table-rules> </rdb:sharding-rule> </rdb:data-source> <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> <property name="dataSource" ref="shardingDataSource" /> </bean> </beans>

  這裡我簡單介紹下一些屬性的含義,

<rdb:strategy id="tableShardingStrategy" sharding-columns="user_id" algorithm-class="com.meiren.member.common.sharding.MemberSingleKeyTableShardingAlgorithm"/>  配置分表規則器  sharding-columns:分表規 則 

  依賴的名(根據user_id取模分表),algorithm-class:分表規則的實現類 

  <rdb:sharding-rule data-sources="dataSource"> 這裡填寫關聯資料來源(多個數據源用逗號隔開),

  <rdb:table-rule logic-table="member_index" actual-tables="member_index_tbl_${[0,1,2,3,4,5,6,7,8,9]}${0..9}"  table-strategy="tableShardingStrategy"/>  logic-table:邏輯表名(mybatis中代替的表名)actual-tables

  資料庫實際的表名,這裡支援inline表示式,比如:member_index_tbl_${0..2}會解析成member_index_tbl_0,member_index_tbl_1,member_index_tbl_2;member_index_tbl_${[a,b,c]}會被解析成

    member_index_tbl_a,member_index_tbl_b和member_index_tbl_c,兩種表示式一起使用的時候,會採取笛卡爾積的方式:member_index_tbl_${[a,b]}${0..2}解析為member_index_tbl_a0,member_index_tbl_a1                                       member_index_tbl_a2,member_index_tbl_b0,member_index_tbl_b1,member_index_tbl_b2;table-strategy:前面定義的分表規則器;

     三、配置好改檔案後,需要修改之前我們的spring-dataSource的幾個地方,把sqlSessionFactory和transactionManager原來關聯的dataSource統一修改為shardingDataSource(這一步作用就是把資料來源全部託管給sharding去管理)

 四、實現分表(分庫)邏輯,我們的分表邏輯類需要實現SingleKeyTableShardingAlgorithm介面的三個方法doBetweenSharding、doEqualSharding、doInSharding

/**
 * 分表邏輯
 * @author zhangwentao
 *
 */
public class MemberSingleKeyTableShardingAlgorithm implements SingleKeyTableShardingAlgorithm<Long> {
    
    /**
     * sql between 規則
     */
    public Collection<String> doBetweenSharding(Collection<String> tableNames, ShardingValue<Long> shardingValue) {
        Collection<String> result = new LinkedHashSet<String>(tableNames.size());
        Range<Long> range = (Range<Long>) shardingValue.getValueRange();
        for (long i = range.lowerEndpoint(); i <= range.upperEndpoint(); i++) {
            Long modValue = i % 100;
            String modStr = modValue < 10 ? "0" + modValue : modValue.toString();
            for (String each : tableNames) {
                if (each.endsWith(modStr)) {
                    result.add(each);
                }
            }
        }
        return result;
    }

    /**
     * sql == 規則
     */
    public String doEqualSharding(Collection<String> tableNames, ShardingValue<Long> shardingValue) {
        Long modValue = shardingValue.getValue() % 100;
        String modStr = modValue < 10 ? "0" + modValue : modValue.toString();
        for (String each : tableNames) {
            if (each.endsWith(modStr)) {
                return each;
            }
        }
        throw new IllegalArgumentException();
    }

    /**
     * sql in 規則
     */
    public Collection<String> doInSharding(Collection<String> tableNames, ShardingValue<Long> shardingValue) {

        Collection<String> result = new LinkedHashSet<String>(tableNames.size());
        for (long value : shardingValue.getValues()) {
            Long modValue = value % 100;
            String modStr = modValue < 10 ? "0" + modValue : modValue.toString();
            for (String tableName : tableNames) {
                if (tableName.endsWith(modStr)) {
                    result.add(tableName);
                }
            }
        }
        return result;
    }
    
}

五、以上四步,我們就完成了sharding-jdbc的搭建,我們可以寫一個測試demo來檢查我們的成果

<select id="getDetailsById" resultType="com.meiren.member.dataobject.MemberDetailsDO"
        parameterType="java.lang.Long">
        select user_id userId ,qq,email from member_details where     user_id =#{userId} limit 1
    </select>
123456789101112131415161718192021private static final String SERVICE_PROVIDER_XML = "/spring/member-service.xml";private static final String BEAN_NAME = "idcacheService";private ClassPathXmlApplicationContext context = null;IdcacheServiceImpl bean = null;IdcacheDao idcacheDao;@Beforepublic void before() {context= new ClassPathXmlApplicationContext(new String[] {SERVICE_PROVIDER_XML});idcacheDao=context.getBean("IdcacheDao", IdcacheDao.class);}@Testpublic void getAllCreditActionTest() {// int id = bean.insertIdcache();Long s=100l;MemberDetailsDO memberDetailsDO=idcacheDao.getDetailsById(s);System.out.println("QQ---------------------"+memberDetailsDO.getQq());}

  列印sql語句,輸出結果:QQ-------------------------------------100,證明成功!

  注意點:這次搭建過程中,我有碰到一個小坑,就是執行的時候會報錯:,官方文件是有解決方案:引入 <context:property-placeholder location="classpath:/member_service.properties" ignore-unresolvable="true" />  ,引入這行程式碼的時候,·必須要要把這邊管理配配置檔案的bean刪除,換句話說,即Spring容器僅允許最多定義一個PropertyPlaceholderConfigurer(或<context:property-placeholder/>),其餘的會被Spring忽略掉(當時搞了半天啊)