MySQL資料庫分片（分庫分表）
阿新 • • 發佈：2019-01-04
分庫分表
將存放在一個數據庫中的資料，按照特定方式進行拆分，分散到多個數據庫中，已達到分散單臺裝置負載的效果
| 垂直分割（縱向切分） | 水平分割（橫向切分） |
|---|---|
| 將單個表，拆分成多個表，分散到不同的資料庫 將單資料庫的多個表進行分類，按照業務類別分散到不同的資料庫上 |
按照表中的某個欄位的某種規則，把表中的許多記錄按行切分，分到多個數據庫中 |
常用軟體mycat（基於阿里Cobar研發的開源軟體）
基於java的分散式資料庫系統中間層，適合資料大量寫入，併為高併發環境的分散式訪問提供解決方案
支援JDBC形式連結，MySQL，Oracle,Sqlserver,Mongodb等
提供資料讀寫分離，分片服務，可以實現資料庫的高可用
| 分片規則 | 1.列舉法 sharding-by-intfile 2.固定分片rule1 3範圍約定auto-sharding-long 4.求模法mod-long 5.日期列分割槽法sharding-by-date |
6.通配取模sharding-by-pattern 7.ASCII碼求模通配sharding-by-prefixpattern 8.編碼指定sharding-by-substring 9.字串拆分hash解析sharding-by-stringhash 10.一致性hash sharding-by-murmur |
|---|
工作過程
mysql收到SQL查詢--解析查詢涉及的表--查看錶的定義，如果有規則，獲取欄位的值，並匹配分片函式，或得列表--將SQL發往分片執行--手機和處理結果資料，返回客戶端
配置mycat
| client 192.168.4.50 |
mycat伺服器 192.168.4.56 |
mysql伺服器 192.168.4.54 |
mysql伺服器 192.168.4.55 |
裝環境（56主機安裝JDK，mycat服務軟體包，54/55為mysql伺服器）
[[email protected] ~]# rpm -qa | grep -i jdk
java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
java-1.8.0-openjdk-headless-1.8.0.131-11.b12.el7.x86_64
copy-jdk-configs-2.2-3.el7.noarch
[[email protected] ~]# yum -y install java-1.8.0-openjdk-devel
[[email protected] ~]# java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)
[[email protected] ~]# tar -zxvf Mycat-server-1.4-beta-20150604171601-linux.tar.gz
[[email protected] ~]# mv mycat /usr/local/
[[email protected] ~]# ls /usr/local/mycat/
bin catlet conf lib logs version.txt
//bin--mycat命令，如啟動，停止 catlet--擴充套件功能 conf--配置檔案 lib--mycat使用的jar包
log--mycat啟動和執行日誌 wrapper.log--mycat服務啟動日誌（執行才會有） mycat.log--記錄SQL指令碼執行的報錯內容（執行才會有）
[[email protected] conf]# ls *.xml
ehcache.xml log4j.xml router.xml rule.xml schema.xml server.xml
//rule.xml 定義mycat分片規則 server.xml 設定連mycat的賬號資訊 schema.xml 配置mycat的真實庫表
[[email protected] mysql]# systemctl restart mysqld
[[email protected] mysql]# systemctl restart mysqld
修改配置檔案server.xml schema.xml
[[email protected] conf]# vim /usr/local/mycat/conf/server.xml
</system>
34 <user name="admin"> //連mycat使用者名稱
35 <property name="password">123456</property> //對應密碼
36 <property name="schemas">TESTDB</property>
[[email protected] conf]# vim /usr/local/mycat/conf/schema.xml
...
5 <schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100"> //schema name名字要和server.xml指定的名字一致
7 <table name="travelrecord" dataNode="dn1,dn2" rule="auto-sharding-long" /> //刪除所有datanode節點（包括後面的）裡的dn3，因為只有兩臺資料伺服器
...
37 <dataNode name="dn1" dataHost="localhost1" database="db1" />
38 <dataNode name="dn2" dataHost="localhost2" database="db2" /> //datahost資料庫主機 database主機對應資料庫
...
46 <writeHost host="hostM1" url="192.168.4.54:3306" user="root"
47 password="123456"> //定義url password要符合資料庫主機許可權
48 <!-- can have multi read hosts -->
49
50 </writeHost>
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1"
slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<writeHost host="hostM1" url="192.168.4.54:3306" user="root"
password="123456">
<!-- can have multi read hosts -->
</writeHost>
<!-- <writeHost host="hostM1" url="localhost:3316" user="root"
password="123456"/> -->
</dataHost> //把<datahost name>..</datahost>複製一份 有多少資料庫伺服器複製多少 並更改對應name url host
在54和55上建立mycat對應庫和使用者，並在56上測試
[[email protected] ~]# mysql -uroot -p123456
mysql> create database db1;
Query OK, 1 row affected (0.00 sec）
mysql> grant all on *.* to [email protected]"%" identified by "123456";
[[email protected] ~]# mysql -uroot -p123456
mysql> create database db2;
Query OK, 1 row affected (0.00 sec）
mysql> grant all on *.* to [email protected]"%" identified by "123456";
[[email protected] conf]# mysql -uroot -h192.168.4.54 -p123456 //在56上驗證使用者對54/55的連結
mysql>exit
[[email protected] conf]# mysql -uroot -h192.168.4.55 -p123456
mysql>修改資料庫伺服器不去分庫名及表名字母大小寫
[[email protected] ~]# vim /etc/my.cnf //55上同操作
[mysqld]
...
lower_case_table_names=1
[[email protected] ~]# systemctl restart mysqld
啟動mycat服務
[[email protected] conf]# /usr/local/mycat/bin/mycat status //檢視狀態
Mycat-server is not running.
[[email protected] conf]# /usr/local/mycat/bin/mycat start //啟動
Starting Mycat-server...
[[email protected] conf]# /usr/local/mycat/bin/mycat status
Mycat-server is running (10799).
[[email protected] conf]# netstat -pntul | grep :8066 //還要再看埠是否啟動
tcp6 0 0 :::8066 :::* LISTEN 10801/java
[[email protected] conf]# ps -C java
PID TTY TIME CMD
10801 ? 00:00:01 java
[[email protected] conf]# ls /usr/local/mycat/logs/ //日誌檔案
mycat.log mycat.pid wrapper.log
[[email protected] conf]# kill -9 10801 //結束程序
[[email protected] conf]# netstat -pntul | grep :8066
[[email protected] conf]# 驗證分片，並在54，和55上檢視
[[email protected] ~]# mysql -h192.168.4.56 -P8066 -uadmin -p123456
mysql> show databases;
+----------+
| DATABASE |
+----------+
| TESTDB |
+----------+
1 row in set (0.01 sec)
mysql> use TESTDB;
mysql> show tables;
+------------------+
| Tables in TESTDB |
+------------------+
| company |
| customer |
| customer_addr |
| employee |
| goods |
| hotnews |
| orders |
| order_items |
| travelrecord |
+------------------+
9 rows in set (0.00 sec)
mysql> create table employee（
-> ID int primary key auto_increment,
-> sharding_id int,
-> name char(15),
-> age tinyint unsigned default 28
-> );
Query OK, 0 rows affected (0.54 sec)
mysql> desc employee ；
+-------------+---------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| sharding_id | int(11) | YES | | NULL | |
| name | char(15) | YES | | NULL | |
| age | tinyint(3) unsigned | YES | | 28 | |
+-------------+---------------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
mysql> insert into employee(sharding_id,name) values(10000,"bob");
Query OK, 1 row affected (0.04 sec)
mysql> insert into employee(sharding_id,name) values(10010,"tom");
Query OK, 1 row affected (0.05 sec)
mysql> select * from db1.employee;
+----+-------------+------+------+
| ID | sharding_id | name | age |
+----+-------------+------+------+
| 1 | 10000 | bob | 28 |
+----+-------------+------+------+
1 row in set (0.00 sec)
mysql> select * from db2.employee;
+----+-------------+------+------+
| ID | sharding_id | name | age |
+----+-------------+------+------+
| 1 | 10010 | tom | 28 |
+----+-------------+------+------+
1 row in set (0.00 sec)