快取穿透

 快取穿透是指使用者查詢資料,在資料庫沒有,自然在快取中也不會有。這樣就導致使用者查詢的時候,在快取中找不到,每次都要去資料庫再查詢一遍,然後返回空。這樣請求就繞過快取直接查資料庫,這也是經常提的快取命中率問題。

解決的辦法就是:如果查詢資料庫也為空,直接設定一個預設值存放到快取,這樣第二次到緩衝中獲取就有值了,而不會繼續訪問資料庫,這種辦法最簡單粗暴。

 

把空結果,也給快取起來,這樣下次同樣的請求就可以直接返回空了,即可以避免當查詢的值為空時引起的快取穿透。同時也可以單獨設定個快取區域儲存空值,對要查詢的key進行預先校驗,然後再放行給後面的正常快取處理邏輯。

 

查詢查不到的資料,在快取中沒有,而直接走了資料庫! 反反覆覆的去這麼做就崩潰了哦

4沒有,redis中沒有,然後去DB查詢,會導致雪崩效應。稱之為 穿透效應。

 

穿透 產生的原因:客戶端隨機生成不同的key,在redis快取中沒有該資料,資料庫也沒有該資料。這樣的話可能導致一直髮生jdbc連線

 

解決方案:

   1、通過閘道器判斷客戶端傳入對應key的規則,不符合資料庫查詢規則,直接返回空 

   2、如果使用的key資料庫查詢不到的話,直接在redis中存一份null結果。 

      在存入id為4的資料庫的時候,直接清除對應redis為4的快取(此時是空哈)

 

廢話不多說,上程式碼:

 pom:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.toov5.architect</groupId>
  <artifactId>architect</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.0.0.RELEASE</version>
	</parent>
	<dependencies>
		<!-- SpringBoot 對lombok 支援 -->
		<dependency>
			<groupId>org.projectlombok</groupId>
			<artifactId>lombok</artifactId>
		</dependency>
		<!-- SpringBoot web 核心元件 -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-tomcat</artifactId>
		</dependency>
		<!-- SpringBoot 外部tomcat支援 -->
		<dependency>
			<groupId>org.apache.tomcat.embed</groupId>
			<artifactId>tomcat-embed-jasper</artifactId>
		</dependency>
		<!-- springboot-log4j -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-log4j</artifactId>
			<version>1.3.8.RELEASE</version>
		</dependency>
		<!-- springboot-aop 技術 -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-aop</artifactId>
		</dependency>
		<!-- https://mvnrepository.com/artifact/commons-lang/commons-lang -->
		<dependency>
			<groupId>commons-lang</groupId>
			<artifactId>commons-lang</artifactId>
			<version>2.6</version>
		</dependency>
		<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
		<dependency>
			<groupId>org.apache.httpcomponents</groupId>
			<artifactId>httpclient</artifactId>
		</dependency>
		<!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>fastjson</artifactId>
			<version>1.2.47</version>
		</dependency>
		<dependency>
			<groupId>javax.servlet</groupId>
			<artifactId>jstl</artifactId>
		</dependency>
		<dependency>
			<groupId>taglibs</groupId>
			<artifactId>standard</artifactId>
			<version>1.1.2</version>
		</dependency>
		<!--開啟 cache 快取 -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-cache</artifactId>
		</dependency>
		<!-- ehcache快取 -->
		<dependency>
			<groupId>net.sf.ehcache</groupId>
			<artifactId>ehcache</artifactId>
			<version>2.9.1</version><!--$NO-MVN-MAN-VER$ -->
		</dependency>
		<dependency>
			<groupId>org.mybatis.spring.boot</groupId>
			<artifactId>mybatis-spring-boot-starter</artifactId>
			<version>1.1.1</version>
		</dependency>
		<!-- mysql 依賴 -->
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
		</dependency>
		<!-- redis 依賴 -->
		<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>
	</dependencies>
  
</project>

 service:

 

package com.toov5.service;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.ehcache.EhCacheCacheManager;
import org.springframework.stereotype.Component;

import net.sf.ehcache.Cache;
import net.sf.ehcache.Element;


@Component
public class EhCacheUtils {

    // @Autowired
    // private CacheManager cacheManager;
    @Autowired
    private EhCacheCacheManager ehCacheCacheManager;

    // 新增本地快取 (相同的key 會直接覆蓋)
    public void put(String cacheName, String key, Object value) {
        Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName);
        Element element = new Element(key, value);
        cache.put(element);
    }

    // 獲取本地快取
    public Object get(String cacheName, String key) {
        Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName);
        Element element = cache.get(key);
        return element == null ? null : element.getObjectValue();
    }

    public void remove(String cacheName, String key) {
        Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName);
        cache.remove(key);
    }

}
package com.toov5.service;

import java.util.Set;
import java.util.concurrent.TimeUnit;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;

@Component
public class RedisService {

    @Autowired
    private StringRedisTemplate stringRedisTemplate;
    //這樣該方法支援多種資料型別 
    public void set(String key , Object object, Long time){
        //開啟事務許可權
        stringRedisTemplate.setEnableTransactionSupport(true);
        try {
            //開啟事務
            stringRedisTemplate.multi();
            
            String argString =(String)object;  //強轉下
            stringRedisTemplate.opsForValue().set(key, argString);
            
            //成功就提交
            stringRedisTemplate.exec();
        } catch (Exception e) {
            //失敗了就回滾
            stringRedisTemplate.discard();
            
        }
        if (object instanceof String ) {  //判斷下是String型別不
            String argString =(String)object;  //強轉下
            //存放String型別的
            stringRedisTemplate.opsForValue().set(key, argString);
        }
        //如果存放Set型別
        if (object instanceof Set) {
            Set<String> valueSet =(Set<String>)object;
            for(String string:valueSet){
                stringRedisTemplate.opsForSet().add(key, string);  //此處點選下原始碼看下 第二個引數可以放好多
            }
        }
        //設定有效期
        if (time != null) {
            stringRedisTemplate.expire(key, time, TimeUnit.SECONDS);
        }
        
    }
    //做個封裝
    public void setString(String key, Object object){
        String argString =(String)object;  //強轉下
        //存放String型別的
        stringRedisTemplate.opsForValue().set(key, argString);
    }
    public void setSet(String key, Object object){
        Set<String> valueSet =(Set<String>)object;
        for(String string:valueSet){
            stringRedisTemplate.opsForSet().add(key, string);  //此處點選下原始碼看下 第二個引數可以放好多
        }
    }
    
    public String getString(String key){
     return    stringRedisTemplate.opsForValue().get(key);
    }
    
    
}
package com.toov5.service;

import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.web.bind.annotation.RequestMapping;

import com.toov5.entity.Users;
import com.toov5.mapper.UserMapper;

import io.netty.util.internal.StringUtil;

@Service
public class SnowslideService {
    @Autowired
    private UserMapper userMapper; 
    @Autowired
    private RedisService redisService;
    
    private Lock lock = new ReentrantLock();
    
    public String getUser01(Long id){
    //定義key, key以當前的類名+方法名+id+引數值
    String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName()
                        + "-id:" + id;    
          //1查詢redis
    String username = redisService.getString(key);
    if (!StringUtil.isNullOrEmpty(username)) {
        return username;
    }
    String resultUsaerName = null;
    try {
        //開啟鎖
        lock.lock();
        Users user = userMapper.getUser(id);
        if (username == null) {
            return null;
        }
        resultUsaerName =user.getName();
        redisService.setString(key, resultUsaerName);
    } catch (Exception e) {
        // TODO: handle exception
    }finally {
        //釋放鎖
        lock.unlock();
    }
          //3直接返回
    return resultUsaerName;
    }
    
    
    
//穿透解決方案    
    public String getUser02(Long id){
    //定義key, key以當前的類名+方法名+id+引數值
    String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName()
                        + "-id:" + id;    
          //1查詢redis
    System.out.println("查詢redis快取"+"key"+key+".resultUserName");
    String username = redisService.getString(key);
    if (!StringUtil.isNullOrEmpty(username)) {
        return username;
    }
    String resultUsaerName = null;
    //如果資料庫中,沒有對應的資料資訊的時候
       System.out.println("查詢資料庫:id"+id);
        Users user = userMapper.getUser(id);
        if (user == null) {
            resultUsaerName="${null}";  //做個標記  客戶端識別到後 提示下吧
            
        }else {
            resultUsaerName=user.getName();
        }
        System.out.println("寫入redis快取"+"key"+key+".resultUserName"+resultUsaerName);
        redisService.setString(key, resultUsaerName);

          //3直接返回
    return resultUsaerName;
    }
    
    
}
package com.toov5.service;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import com.alibaba.fastjson.JSONObject;
import com.toov5.entity.Users;
import com.toov5.mapper.UserMapper;


import io.netty.util.internal.StringUtil;

@Component
public class UserService {
    @Autowired
    private EhCacheUtils ehCacheUtils;
    @Autowired
    private RedisService redisService;
    @Autowired
    private UserMapper userMapper;
    //定義個全域性的cache名字
    private String cachename ="userCache";
    
    public Users getUser(Long id){
        //先查詢一級快取  key以當前的類名+方法名+id+引數值
        String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName()
                + "-id:" + id;
        //查詢一級快取資料有對應值的存在 如果有 返回
        Users user = (Users)ehCacheUtils.get(cachename, key);
        if (user != null) {
            System.out.println("key"+key+",直接從一級快取獲取資料"+user.toString());
            return user;
        }
        //一級快取沒有對應的值存在,接著查詢二級快取    
        // redis存物件的方式  json格式 然後反序列號
        String userJson = redisService.getString(key);
        //如果rdis快取中有這個對應的值,修改一級快取    最下面的會有的 相同會覆蓋的    
        if (!StringUtil.isNullOrEmpty(userJson)) {  //有 轉成json
            JSONObject jsonObject = new JSONObject();//用的fastjson
            Users resultUser = jsonObject.parseObject(userJson,Users.class);
            ehCacheUtils.put(cachename, key, resultUser);
            return resultUser;
        }
        //都沒有 查詢DB 
        Users user1 = userMapper.getUser(id);
        if (user1 == null) {
            return null;
        }
        //保證兩級快取有效期相同!?   一級快取時間-二級快取執行的時間
        //一級快取時間 等於 二級快取剩下的時間   
        //存放到二級快取 redis中
        redisService.setString(key, new JSONObject().toJSONString(user1));
        //存放到一級快取 Ehchache
        ehCacheUtils.put(cachename, key, user1);
        return user1;
    }
    
    
    
}

mapper

package com.toov5.mapper;

import java.util.List;

import org.apache.ibatis.annotations.Param;
import org.apache.ibatis.annotations.Select;
import org.springframework.cache.annotation.CacheConfig;
import org.springframework.cache.annotation.Cacheable;

import com.toov5.entity.Users;
//引入的jar包後就有了這個註解了 非常好用 (配置快取的基本資訊)
@CacheConfig(cacheNames={"userCache"})  //快取的名字  整個類的
public interface UserMapper {
    @Select("SELECT ID ,NAME,AGE FROM users where id=#{id}")
    @Cacheable //讓這個方法實現快取 查詢完畢後 存入到快取中  不是每個方法都需要快取呀!save()就不用了吧
    Users getUser(@Param("id") Long id);
}

entity

package com.toov5.entity;

import java.io.Serializable;

import lombok.Data;

@Data
public class Users implements Serializable{
  private String name;
  private Integer age;
}

controller

package com.toov5.controller;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import com.toov5.service.SnowslideService;

@RestController
public class UserRedisController {
   @Autowired
   private SnowslideService snowslideService;
   
   @RequestMapping("/getUser02")
   public String getUser02(Long id){
       return snowslideService.getUser02(id); 
   }
   
}
package com.toov5.controller;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import com.toov5.entity.Users;
import com.toov5.service.UserService;

@RestController
public class IndexController {
    @Autowired
    private UserService userService;
    
    @RequestMapping("/userId")
    public Users getUserId(Long id){
        return userService.getUser(id);  
    }
    
   
}

啟動類

package com.toov5.app;

import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;

@EnableCaching //開啟快取
@MapperScan(basePackages={"com.toov5.mapper"})
@SpringBootApplication(scanBasePackages={"com.toov5.*"})
public class app {
   public static void main(String[] args) {
    SpringApplication.run(app.class, args);
}
    
}

yml

###埠號配置
server:
  port: 8080
###資料庫配置 
spring:
  datasource:
    url: jdbc:mysql://localhost:3306/test
    username: root
    password: root
    driver-class-name: com.mysql.jdbc.Driver
    test-while-idle: true
    test-on-borrow: true
    validation-query: SELECT 1 FROM DUAL
    time-between-eviction-runs-millis: 300000
    min-evictable-idle-time-millis: 1800000
# 快取配置讀取
  cache:
    type: ehcache
    ehcache:
      config: classpath:app1_ehcache.xml
  redis:
    database: 0 
    jedis:
      pool:
        max-active: 8
        max-wait: -1
        max-idle: 8
        min-idle: 0
    timeout: 10000
    cluster:
      nodes:
        - 192.168.91.5:9001
        - 192.168.91.5:9002
        - 192.168.91.5:9003
        - 192.168.91.5:9004
        - 192.168.91.5:9005
        - 192.168.91.5:9006

 

<?xml version="1.0" encoding="UTF-8"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd">

	<diskStore path="java.io.tmpdir/ehcache-rmi-4000" />


	<!-- 預設快取 -->
	<defaultCache maxElementsInMemory="1000" eternal="true"
		timeToIdleSeconds="120" timeToLiveSeconds="120" overflowToDisk="true"
		diskSpoolBufferSizeMB="30" maxElementsOnDisk="10000000"
		diskPersistent="true" diskExpiryThreadIntervalSeconds="120"
		memoryStoreEvictionPolicy="LRU">
	</defaultCache>
  
	<!-- demo快取 --><!-- name="userCache" 對應我們在 @CacheConfig(cacheNames={"userCache"}) !!!!! -->
	<!--Ehcache底層也是用Map集合實現的 -->
	<cache name="userCache" maxElementsInMemory="1000" eternal="false"
		timeToIdleSeconds="120" timeToLiveSeconds="120" overflowToDisk="true"
		diskSpoolBufferSizeMB="30" maxElementsOnDisk="10000000"
		diskPersistent="false" diskExpiryThreadIntervalSeconds="120"
		memoryStoreEvictionPolicy="LRU">  <!-- LRU快取策略 -->
		<cacheEventListenerFactory
			class="net.sf.ehcache.distribution.RMICacheReplicatorFactory" />
		<!-- 用於在初始化快取,以及自動設定 -->
		<bootstrapCacheLoaderFactory
			class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory" />
	</cache>
</ehcache>

 再加一個攔截

 

執行結果:

 

 

把空結果,也給快取起來,這樣下次同樣的請求就可以直接返回空了,即可以避免當查詢的值為空時引起的快取穿透。同時也可以單獨設定個快取區域儲存空值,對要查詢的key進行預先校驗,然後再放行給後面的正常快取處理邏輯。

 

注意:再給對應的ip存放真值的時候,需要先清除對應的之前的空快取。

 

補充熱點key

 

熱點key:某個key訪問非常頻繁,當key失效的時候有打量執行緒來構建快取,導致負載增加,系統崩潰。

 

解決辦法:

①使用鎖,單機用synchronized,lock等,分散式用分散式鎖。

②快取過期時間不設定,而是設定在key對應的value裡。如果檢測到存的時間超過過期時間則非同步更新快取。

③在value設定一個比過期時間t0小的過期時間值t1,當t1過期的時候,延長t1並做更新快取操作。