1. 程式人生 > >MyBatis動態數據源配置

MyBatis動態數據源配置

== word release cycle res 指定 靜態 從庫 pointer

1.通過spirngboot構建項目
主要依賴 mybatis+mysql+aspect

<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>

<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>1.8.7</version>
</dependency>
2.在application.properties配置數據庫
#主庫
datasource.master.driverClassName=com.mysql.jdbc.Driver
datasource.master.url=jdbc:mysql://ip:3306/dbname?characterEncoding=UTF-8
datasource.master.username=
datasource.master.password=

#從庫
datasource.slave.driverClassName=com.mysql.jdbc.Driver
datasource.slave.url=jdbc:mysql://ip:3306/dbname?characterEncoding=UTF-8
datasource.slave.username=
datasource.slave.password=


3.在啟動類關閉自動配置數據源
這個很重要,如果沒有關閉,在啟動程序時會發生循環依賴的錯誤

@EnableAutoConfiguration(exclude = {DataSourceAutoConfiguration.class})@SpringBootApplication
br/>@SpringBootApplication

public static void main(String[] args) {
    SpringApplication.run(DynamicDatasourceAnnoApplication.class, args);
}

}
4.配置數據源
要實現數據源的動態選擇,AbstractRoutingDataSource這個抽象類很重要,通過閱讀源碼,發現數據源在該類中以key-value的形式存儲在map中,其中在determineTargetDataSource()方法中,通過determineCurrentLookupKey()方法得到key,在map中尋找需要的數據源,而determineCurrentLookupKey()是抽象方法,我們可以通過繼承這個抽象類實現該方法,配置我們期望的數據源。

public abstract class AbstractRoutingDataSource extends AbstractDataSource implements InitializingBean {
protected DataSource determineTargetDataSource() {
Assert.notNull(this.resolvedDataSources, "DataSource router not initialized");
Object lookupKey = this.determineCurrentLookupKey();
DataSource dataSource = (DataSource)this.resolvedDataSources.get(lookupKey);
if (dataSource == null && (this.lenientFallback || lookupKey == null)) {
dataSource = this.resolvedDefaultDataSource;
}

    if (dataSource == null) {
        throw new IllegalStateException("Cannot determine target DataSource for lookup key [" + lookupKey + "]");
    } else {
        return dataSource;
    }

}

protected abstract Object determineCurrentLookupKey();
在了解了基本原理後,開始配置數據源,繼承AbstractRoutingDataSource 類
DynamicDataSource
public class DynamicDataSource extends AbstractRoutingDataSource {@Override
br/>@Override
return DynamicDataSourceHolder.getType();
}
}
實現了該方法後,如何返回一個key,這個key從何而來?
每個請求都是一個線程,我們可以再創建一個類,其中使用ThreadLocal線程局部變量保存每個線程所請求方法對應的key,通過該類的靜態方法返回該請求的key
DynamicDataSourceHolder
public class DynamicDataSourceHolder {
private static final ThreadLocal<DataSourceType> dataSourceHolder = new ThreadLocal<>();

private static final Set<DataSourceType> dataSourceTypes = new HashSet<>();
static {
    dataSourceTypes.add(DataSourceType.MASTER);
    dataSourceTypes.add(DataSourceType.SLAVE);
}

public static void setType(DataSourceType dataSourceType){
    if(dataSourceType == null){
        throw new NullPointerException();
    }
    dataSourceHolder.set(dataSourceType);
}

public static DataSourceType getType(){
    return dataSourceHolder.get();
}

public static void clearType(){
    dataSourceHolder.remove();
}

public static boolean containsType(DataSourceType dataSourceType){
    return dataSourceTypes.contains(dataSourceType);
}

}
其中使用了枚舉表示不同類型的數據源
DataSourceType

public enum DataSourceType {
MASTER,
SLAVE,
}
創建了關鍵類DynamicDataSource後,開始配置數據源

DynamicDataSourceConfig

@Configuration
public class DynamicDataSourceConfig {

@Bean(name = "masterDataSource")
@ConfigurationProperties(prefix = "datasource.master")
public DataSource masterDataSource(){
    return DataSourceBuilder.create().build();
}

@Bean(name = "slaveDataSource")
@ConfigurationProperties(prefix = "datasource.slave")
public DataSource slaveDataSource(){
    return DataSourceBuilder.create().build();
}

@Bean(name = "dynamicDataSource")
@Primary
public DataSource dynamicDataSource(@Qualifier(value = "masterDataSource") DataSource masterDataSource,
                                    @Qualifier(value = "slaveDataSource") DataSource slaveDataSource){
    DynamicDataSource dynamicDataSource = new DynamicDataSource();
    dynamicDataSource.setDefaultTargetDataSource(masterDataSource);
    Map<Object,Object> dataSourceMap = new HashMap<>();
    dataSourceMap.put(DataSourceType.MASTER,masterDataSource);
    dataSourceMap.put(DataSourceType.SLAVE,slaveDataSource);
    dynamicDataSource.setTargetDataSources(dataSourceMap);
    return dynamicDataSource;
}

}
配置好數據源後,開始配置mybatis

MyBatisConfig

@Configuration
@MapperScan(basePackages = "com.example.dynamic_datasource_anno.db.dao")
public class MyBatisConfig {

@Autowired
private DataSource dataSource;

@Bean(name = "sqlSessionFactory")
public SqlSessionFactory sqlSessionFactory() throws Exception {
    SqlSessionFactoryBean sessionFactoryBean = new SqlSessionFactoryBean();
    sessionFactoryBean.setDataSource(dataSource);
    sessionFactoryBean.setTypeAliasesPackage("com.example.dynamic_datasource_anno.db.pojo");
    Resource[] resources = new PathMatchingResourcePatternResolver()
            .getResources("classpath:com/example/dynamic_datasource_anno/db/mapper/*Mapper.xml");
    sessionFactoryBean.setMapperLocations(resources);

    return sessionFactoryBean.getObject();
}

}
數據源和mybatis配置好後,開始配置切面,動態地選擇都在此類中完成。

第一種 Anno+Aspect
創建註解DBType,默認為主數據庫

@Retention(RetentionPolicy.RUNTIME)@Target(ElementType.METHOD)
br/>@Target(ElementType.METHOD)
DataSourceType value() default DataSourceType.MASTER;}
切面DynamicDataSourceAspect
@Aspect
br/>}
切面DynamicDataSourceAspect
@Aspect
public class DynamicDataSourceAspect {

private static final Logger logger = LoggerFactory.getLogger(DynamicDataSourceAspect.class);

@Before("@annotation(dbType)")
public void changeDataSourceType(JoinPoint joinPoint, DBType dbType){
    DataSourceType curType = dbType.value();
    if(!DynamicDataSourceHolder.containsType(curType)){
        logger.info("指定數據源[{}]不存在,使用默認數據源-> {}",dbType.value(),joinPoint.getSignature());
    }else{
        logger.info("use datasource {} -> {}",dbType.value(),joinPoint.getSignature());
        DynamicDataSourceHolder.setType(dbType.value());
    }

}

@After("@annotation(dbType)")
public void restoreDataSource(JoinPoint joinPoint, DBType dbType){
    logger.info("use datasource {} -> {}",dbType.value(),joinPoint.getSignature());
    DynamicDataSourceHolder.clearType();
}

}
配置好切面後,可以在Mapper接口的方法上使用@DBType指定要使用的數據源,使用自動代碼生成插件將表中數據生成對應的實體類,mapper文件,mapper接口

@DBType(DataSourceType.SLAVE)
public void select(){
User user = userMapper.selectByPrimaryKey(1);
System.out.println(user.getId() + "--" + user.getName() + "==" + user.getGender());
}

@DBType(DataSourceType.MASTER)
public void insert(User user){
userMapper.insertSelective(user);
}
第二種 只使用Aspect,判斷sql類型
在此方式中,沒有再使用枚舉、註解,數據源和mybatis的配置與之前相似,不再贅述,詳細配置可以查看此方式的源碼。主要不同在於切面的配置,此配置中通過切點判斷請求的SqlCommandType來選擇數據源
DynamicDataSourceAspect

@Aspect@Component
br/>@Component

private static final Logger logger = LoggerFactory.getLogger(DynamicDataSourceAspect.class);
@Autowired
@Qualifier("sqlSessionFactory")
private SqlSessionFactory sqlSessionFactory;

// first * any return value
// second * any class name of the dao package
// third * any method name of the class
// (..) any number of parameters
// @Pointcut("within(com.example.dynamic_datasource_method.db.dao..*)")
@Pointcut("execution(* com.example.dynamic_datasource_method.db.dao.*.*(..))")
public void declareJoinPoint(){
}

@Before("declareJoinPoint()")
public void matchDataSoruce(JoinPoint point){

    //得到被代理對象
    Object target = point.getTarget();
    //目標方法名
    String methodName = point.getSignature().getName();

    Class<?>[] interfaces = target.getClass().getInterfaces();
    Class<?>[] parametersTypes = ((MethodSignature)point.getSignature()).getMethod().getParameterTypes();

    try {
        Method method = interfaces[0].getMethod(methodName,parametersTypes);
        String key = interfaces[0].getName() + "." + method.getName();
        //sql 的類型
        SqlCommandType type = sqlSessionFactory.getConfiguration().getMappedStatement(key).getSqlCommandType();
        //查詢從庫 增刪改主庫
        if(type == SqlCommandType.SELECT){
            logger.info("use slaveDataSource");
            DynamicDataSourceHolder.setType("slave");
        }
        if(type == SqlCommandType.DELETE || type == SqlCommandType.UPDATE || type == SqlCommandType.INSERT){
            logger.info("use masterDataSource");
            DynamicDataSourceHolder.setType("master");
        }
    } catch (NoSuchMethodException e) {
        e.printStackTrace();
    }
}

@After("declareJoinPoint()")
public void restoreDataSource(JoinPoint joinPoint){
    logger.info(DynamicDataSourceHolder.getType() + " change to master" + joinPoint.getSignature());
    DynamicDataSourceHolder.clearType();
}

}
通過此方式,可以直接調用mapper接口的方法而不需要註解直接動態地選擇數據源

問題
當在啟動類沒有關閉數據源的自動配置時,啟動項目會報以下錯誤:

Description:

The dependencies of some of the beans in the application context form a cycle:

dynamicDataSourceAspect (field private org.apache.ibatis.session.SqlSessionFactory com.example.dynamic_datasource_method.config.mybatis.DynamicDataSourceAspect.sqlSessionFactory)

myBatisConfig (field private javax.sql.DataSource com.example.dynamic_datasource_method.config.mybatis.MyBatisConfig.dataSource)
┌─────┐
| dynamicDataSource defined in class path resource [com/example/dynamic_datasource_method/config/mybatis/DynamicDataSourceConfig.class]
↑ ↓
| masterDataSource defined in class path resource [com/example/dynamic_datasource_method/config/mybatis/DynamicDataSourceConfig.class]
↑ ↓
| dataSourceInitializer
└─────┘

2018-09-08 10:04:09.537 ERROR 20372 --- [ main] o.s.test.context.TestContextManager : Caught exception while allowing TestExecutionListener [org.springframework.test.context.web.ServletTestExecutionListener@68ceda24] to prepare test instance [com.example.dynamic_datasource_method.DynamicDatasourceMethodApplicationTests@65ae095c]

java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:83) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:189) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:131) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:230) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.junit4.SpringJUnit4Claunner.createTest(SpringJUnit4Claunner.java:228) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.junit4.SpringJUnit4Claunner$1.runReflectiveCall(SpringJUnit4Claunner.java:287) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12]
at org.springframework.test.context.junit4.SpringJUnit4Claunner.methodBlock(SpringJUnit4Claunner.java:289) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.junit4.SpringJUnit4Claunner.runChild(SpringJUnit4Claunner.java:247) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.junit4.SpringJUnit4Claunner.runChild(SpringJUnit4Claunner.java:94) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) [junit-4.12.jar:4.12]
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.junit.runners.ParentRunner.run(ParentRunner.java:363) [junit-4.12.jar:4.12]
at org.springframework.test.context.junit4.SpringJUnit4Claunner.run(SpringJUnit4Claunner.java:191) [spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.junit.runner.JUnitCore.run(JUnitCore.java:137) [junit-4.12.jar:4.12]
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) [junit-rt.jar:na]
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) [junit-rt.jar:na]
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) [junit-rt.jar:na]
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) [junit-rt.jar:na]
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name ‘dynamicDataSourceAspect‘: Unsatisfied dependency expressed through field ‘sqlSessionFactory‘; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name ‘myBatisConfig‘: Unsatisfied dependency expressed through field ‘dataSource‘; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name ‘dynamicDataSource‘ defined in class path resource [com/example/dynamic_datasource_method/config/mybatis/DynamicDataSourceConfig.class]: Unsatisfied dependency expressed through method ‘dynamicDataSource‘ parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘masterDataSource‘ defined in class path resource [com/example/dynamic_datasource_method/config/mybatis/DynamicDataSourceConfig.class]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘dataSourceInitializer‘: Invocation of init method failed; nested exception is org.springframework.beans.factory.BeanCurrentlyInCreationException: Error creating bean with name ‘dynamicDataSource‘: Requested bean is currently in creation: Is there an unresolvable circular reference?
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:588) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1272) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) ~[spring-beans-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) ~[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) ~[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693) ~[spring-boot-1.5.16.BUILD-20180902.070551-14.jar:1.5.16.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360) ~[spring-boot-1.5.16.BUILD-20180902.070551-14.jar:1.5.16.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) ~[spring-boot-1.5.16.BUILD-20180902.070551-14.jar:1.5.16.BUILD-SNAPSHOT]
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:121) ~[spring-boot-test-1.5.16.BUILD-20180902.070551-14.jar:1.5.16.BUILD-SNAPSHOT]
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:98) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:116) ~[spring-test-4.3.18.RELEASE.jar:4.3.18.RELEASE]
... 24 common frames omitted
思考
關於最後一種方式判斷sql類型達到數據源的動態地選擇,此時mysql為一主一從的結構,若為一主多從的結構時如何實現動態選擇?以下是大體思路:

本項目實現了一主一從動態數據源的選擇,使用切面判斷sql方法的方式。

這裏的多數據的動態選擇,主要是一主一從的形式,若有一主多從的方式,怎麽實現數據源的動態選擇?

增刪改可以使用一個主數據庫,查則可以有多個從數據庫可以選擇,對於多個從數據庫,如何選擇?

當有多個從數據庫時,初步設想使用負載均衡,使用輪詢的方式將查詢分布到多個從數據庫中,那如何實現?

動態數據源的實現是繼承了抽象類AbstractRoutingDataSource,實現了其中的抽象方法 determineCurrentLookupKey()
通過方法名可知,該方法返回一個key,通過閱讀AbstractRoutingDataSource的源碼,多個數據源的存儲方式
是以key-value的形式存儲在map中,當動態選擇時,使用determineTargetDataSourc()根據key找到對應的DataSource。

現在要實現多個數據庫,設想繼承AbstractRoutingDataSource類,重寫determineTargetDataSource()方法,將原來Map<Object, Object> targetDataSources的存儲方式改為Map<Object,List<Object>> slaveDataSources 用來存儲從庫數據源,當動態選擇時,在determineTargetDataSource()方法中 判斷 determineCurrentLookupKey()方法的key,若為master,則直接返回resolvedDefaultDataSource,(前提是在配置數據源時,必須將主數據設置為defaultDataSource).若為slave,則得到slave的List<DataSource>,使用輪詢的方式得到其中一個DataSource返回,實現多從的動態選擇。

MyBatis動態數據源配置