可實現的全域性唯一有序ID生成策略
阿新 • • 發佈：2019-10-25
在部落格園搜素全域性唯一有序ID，羅列出來的文章大致講述了以下幾個問題，常見的生成全域性唯一id的常見方法 ：使用資料庫自動增長序列實現 ； 使用UUID實現； 使用redis實現； 使用Twitter的snowflake演算法實現；使用資料庫+本地快取實現。作為一個記錄性質的部落格，簡單總結一下。
在實際的生產場景中，經常會出現如下的情況比方說訂單號：D channelNo 流水號 樣例PSDK1600000001, PSDK1600000002, PSDK1600000003... 這種具有業務意義的全域性唯一id且有序自增。先來看一下使用比較多的Twitter的snowflake演算法，snowflake生成的ID整體上按照時間自增排序，並且整個分散式系統內不會產生ID碰撞（由datacenter和workerId作區分），並且效率較高。經測試snowflake每秒能夠產生26萬個ID。一個簡單的實現如下：
/**
* Twitter的分散式自增ID雪花演算法snowflake
**/
public class SnowFlake {
/**
* 起始的時間戳
*/
private final static long START_STMP = 1480166465631L;
/**
* 每一部分佔用的位數
*/
private final static long SEQUENCE_BIT = 12; //序列號佔用的位數
private final static long MACHINE_BIT = 5; //機器標識佔用的位數
private final static long DATACENTER_BIT = 5;//資料中心佔用的位數
/**
* 每一部分的最大值
*/
private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
/**
* 每一部分向左的位移
*/
private final static long MACHINE_LEFT = SEQUENCE_BIT;
private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT;
private long datacenterId; //資料中心
private long machineId; //機器標識
private long sequence = 0L; //序列號
private long lastStmp = -1L;//上一次時間戳
public SnowFlake(long datacenterId, long machineId) {
if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) {
throw new IllegalArgumentException("datacenterId can't be greater than MAX_DATACENTER_NUM or less than 0");
}
if (machineId > MAX_MACHINE_NUM || machineId < 0) {
throw new IllegalArgumentException("machineId can't be greater than MAX_MACHINE_NUM or less than 0");
}
this.datacenterId = datacenterId;
this.machineId = machineId;
}
/**
* 產生下一個ID
*
* @return
*/
public synchronized long nextId() {
long currStmp = getNewstmp();
if (currStmp < lastStmp) {
throw new RuntimeException("Clock moved backwards. Refusing to generate id");
}
if (currStmp == lastStmp) {
//相同毫秒內，序列號自增
sequence = (sequence + 1) & MAX_SEQUENCE;
//同一毫秒的序列數已經達到最大
if (sequence == 0L) {
currStmp = getNextMill();
}
} else {
//不同毫秒內，序列號置為0
sequence = 0L;
}
lastStmp = currStmp;
return (currStmp - START_STMP) << TIMESTMP_LEFT //時間戳部分
| datacenterId << DATACENTER_LEFT //資料中心部分
| machineId << MACHINE_LEFT //機器標識部分
| sequence; //序列號部分
}
private long getNextMill() {
long mill = getNewstmp();
while (mill <= lastStmp) {
mill = getNewstmp();
}
return mill;
}
private long getNewstmp() {
return System.currentTimeMillis();
}
public static void main(String[] args) {
SnowFlake snowFlake = new SnowFlake(1, 1);
long start = System.currentTimeMillis();
for (int i = 0; i < 10; i++) {
System.out.println(snowFlake.nextId());
}
System.out.println(System.currentTimeMillis() - start);
}
}演算法中引入了時間因子，所以可以保證生成的id唯一且有序，但是滿足不了業務欄位+流水號有序自增的要求。如果在此基礎上再配合使用資料庫本地快取自然也是可以實現的，不過複雜化了。上述程式碼執行兩次結果如下：
385063405393940480 385063405393940481 385063405393940482 385063405393940483 385063405393940484 385063405393940485 385063405393940486 385063405393940487 385063405398134784 385063405398134785 385064572152844288 385064572152844289 385064572152844290 385064572152844291 385064572152844292 385064572152844293 385064572152844294 385064572152844295 385064572152844296 385064572152844297
簡單的方法就是我們放棄自己造輪子的思想。mongodb中資料的基本單元稱為document，在一個特定集合內部需要唯一的標識文件，因此mongdb中儲存的文件都由一個‘_id’鍵，這個鍵的值可以是任意型別的ObjectId，要求不同的機器都能用全域性唯一的同種方法方便的生成它。因此不能使用自增主鍵，ObjectId 底層也是借鑑了雪花演算法，使用12位元組的儲存空間 |0|1|2|3|4|5|6 |7|8|9|10|11| |時間戳 |機器ID|PID|計數器| 前四個位元組時間戳是從標準紀元開始的時間戳，單位為秒 。時間戳保證秒級唯一，機器ID保證設計時考慮分散式，避免時鐘同步，PID保證同一臺伺服器執行多個mongod例項時的唯一性，最後的計數器保證同一秒內的唯一性。
mongo在spring boot中的引入和配置，此處不再介紹。
建立model類
package com.slowcity.admin.generate.dbmodel;
import java.io.Serializable;
public class BaseSequence implements Serializable{
private static final long serialVersionUID = 475722757687764546L;
private String id;
private String name;
private Long sequence;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Long getSequence() {
return sequence;
}
public void setSequence(Long sequence) {
this.sequence = sequence;
}
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((id == null) ? 0 : id.hashCode());
result = prime * result + ((name == null) ? 0 : name.hashCode());
result = prime * result + ((sequence == null) ? 0 : sequence.hashCode());
return result;
}
@Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
BaseSequence other = (BaseSequence) obj;
if (id == null) {
if (other.id != null)
return false;
} else if (!id.equals(other.id))
return false;
if (name == null) {
if (other.name != null)
return false;
} else if (!name.equals(other.name))
return false;
if (sequence == null) {
if (other.sequence != null)
return false;
} else if (!sequence.equals(other.sequence))
return false;
return true;
}
@Override
public String toString() {
return "BaseSequence [id=" + id + ", name=" + name + ", sequence=" + sequence + "]";
}
}
public class DigitalTaskSequence extends BaseSequence{
private static final long serialVersionUID = -7287622688931253780L;
}
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.stereotype.Component;
@Component
@Document(collection = "dm_id_task")
public class DigitalTaskSequenceMG extends DigitalTaskSequence {
private static final long serialVersionUID = -425011291271386371L;
@Id
@Override
public String getId() {
return super.getId();
}
}
service
import java.util.List;
import com.slowcity.admin.generate.dbmodel.BaseSequence;
public interface SequenceGenericService {
public String generateId(Class<? extends BaseSequence> clazz);
List<BaseSequence> initAllId();
}
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.dbmodel.DigitalTaskSequenceMG;
import com.slowcity.admin.generate.repository.SequenceGenericRepository;
import com.slowcity.admin.generate.service.SequenceGenericService;
@Service
@Transactional
public class SequenceGenericServiceImpl implements SequenceGenericService {
private static final Logger log = LoggerFactory.getLogger(SequenceGenericServiceImpl.class);
private SequenceGenericRepository sequenceGenericRepository;
public SequenceGenericServiceImpl(SequenceGenericRepository sequenceGenericRepository) {
this.sequenceGenericRepository = sequenceGenericRepository;
}
@Override
public String generateId(Class<? extends BaseSequence> clazz) {
String id = sequenceGenericRepository.generateId(clazz);
log.info("{} generate {}", clazz.getName(), id);
return id;
}
@Override
public List<BaseSequence> initAllId() {
List<BaseSequence> baseSequenceList = new ArrayList<>(),
baseSequenceResultList = new ArrayList<>();
DigitalTaskSequenceMG digitalTaskSequenceMG = new DigitalTaskSequenceMG();
digitalTaskSequenceMG.setName("sequence");
digitalTaskSequenceMG.setSequence(1210000000000000000L); //1210可以代表業務號 000000000000000代表自增流水號
baseSequenceList.add(digitalTaskSequenceMG);
for (BaseSequence baseSequence:baseSequenceList) {
BaseSequence resultSequence = sequenceGenericRepository.initAllId(baseSequence);
if(resultSequence != null){
baseSequenceResultList.add(resultSequence);
}
}
return baseSequenceResultList;
}
}
資料實現層
import com.slowcity.admin.generate.dbmodel.BaseSequence;
public interface SequenceGenericRepository {
public String generateId(Class<? extends BaseSequence> clazz);
BaseSequence initAllId(BaseSequence Sequence);
}
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.Update;
import org.springframework.stereotype.Component;
import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.repository.SequenceGenericRepository;
@Component
public class SequenceMongoGenericRepository implements SequenceGenericRepository {
private Map<Class,Class<? extends BaseSequence>> baseSequenceMap;
private MongoTemplate mongoTemplate;
public SequenceMongoGenericRepository(List<BaseSequence> baseSequences, MongoTemplate mongoTemplate){
baseSequenceMap = baseSequences.stream()
.collect(Collectors.toMap(baseSequence -> baseSequence.getClass().getSuperclass(),
BaseSequence::getClass));
this.mongoTemplate = mongoTemplate;
}
@Override
public String generateId(Class<? extends BaseSequence> clazz) {
Class<? extends BaseSequence> childClazz = baseSequenceMap.get(clazz);
if(childClazz != null) {
Query query = new Query(Criteria.where("name").is("sequence"));
Update update = new Update().inc("sequence", 1);
Object dbm = mongoTemplate.findAndModify(query, update, childClazz);
if(dbm != null) {
BaseSequence bs = (BaseSequence)dbm;
return String.valueOf(bs.getSequence());
}
}
return null;
}
@Override
public BaseSequence initAllId(BaseSequence Sequence) {
Query query = new Query(Criteria.where("name").is("sequence"));
Class clazz = Sequence.getClass();
List<? extends BaseSequence> list = mongoTemplate.find(query,clazz);
if(list.isEmpty()){
mongoTemplate.save(Sequence);
return Sequence;
}
return null;
}
}
controller
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.dbmodel.DigitalTaskSequence;
import com.slowcity.admin.generate.service.SequenceGenericService;
/**
* id生成器
* @author moona
*
*/
@RestController
@RequestMapping("/generateId/task")
public class TaskGenerateIdController {
@Autowired
private SequenceGenericService sequenceGenericService;
@RequestMapping(value = "/taskId", method = RequestMethod.GET)
public String generateTaskId() {
return sequenceGenericService.generateId(DigitalTaskSequence.class);
}
@RequestMapping(value = "/init", method = RequestMethod.GET)
public List<BaseSequence> generateTaskIdinit() {
return sequenceGenericService.initAllId();
}
}
執行初始化呼叫方法
對應資料庫
開始測試生成id
第一次呼叫：
第2次呼叫
第10次呼叫
此時再檢視資料庫，序列已經到1210000000000000011 下次呼叫直接取值了。真正做到了了分散式滿足業務的自增全域性唯一索引。mongo底層是原子性的，所以也不會出現併發的問題。如果將id生成策略部署成單臺機器服務，則可以滿足不同服務不同業務的需求，真正做到可定製可擴充套件。儘可放心使用。
【end】
&n