MySQL 之 MHA + ProxySQL + keepalived 實現讀寫分離，高可用（二）
ProxySQL安裝
yum/rpm安裝
在github或官網上可以下載rpm包，wiki的Getting start章節有詳細介紹。
cat <<EOF | tee /etc/yum.repos.d/proxysql.repo [proxysql_repo] name= ProxySQL YUM repository baseurl=http://repo.proxysql.com/ProxySQL/proxysql-1.4.x/centos/7 gpgcheck=1 gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key EOF yum clean all yum makecache yum install proxysql
服務管理
systemctl start proxysql systemctl stop proxysql systemctl status proxysql
檢視啟動檔案以及配置檔案位置
rpm -ql proxysql /etc/init.d/proxysql #proxysql的啟動控制檔案 /etc/proxysql.cnf #proxysql配置檔案 /usr/bin/proxysql /usr/share/proxysql/tools/proxysql_galera_checker.sh /usr/share/proxysql/tools/proxysql_galera_writer.pl
修改配置檔案
配置檔案層級
簡單說就是配置proxysql分為三個級別，RUNTIME是即時生效的，MEMORY是儲存在記憶體中但並不立即生效的，DISK|CONFIG FILE是持久化或寫在配置檔案中的。
這三個級別的配置檔案互不干擾，在某個層級修改了配置檔案，想要載入或儲存到另一個層級，需要額外的LOAD或SAVE操作：LOAD xx_config FROM xx_level | LOAD xx_config TO xx_level | SAVE xx_config TO xx_level | SAVE xx_config FROM xx_level，達到載入配置或者持久化配置的目的。
RUNTIME層級的配置時在proxysql管理庫(sqlite)的main庫中以runtime_開頭的表，這些表的資料庫無法直接修改，只能從其他層級載入；
MEMORY層級的配置在main庫中以mysql_開頭的表以及global_variables表，這些表的資料可以直接修改；
DISK|CONFIG FILR層級的配置在磁碟上的sqlite庫或配置檔案裡。
配置檔案的修改流程一般是：
啟動時：先修改必要的CONFIG FILE配置，比如管理埠，然後啟動；
其他配置：修改MEMORY中的表，然後載入到RUNTIME並持久化。
更多資訊：Configuring ProxySQL
mysql_ifaces
First of all, bear in mind that the best way to configure ProxySQL is through its admin interface. This lends itself to online configuration (without having to restart the proxy) via SQL queries to its admin database. It’s an effective way to configure it both manually and in an automated fashion.
As a secondary way to configure it, we have the configuration file.
也就是說proxysql有一個admin介面專門來做配置，相當於一個mysql shell可以通過sql來讓配置實時生效。
mysql_ifaces配置了允許連線proxysql的ip和port：
vi /etc/proxysql.cnf
# 將admin_variables中的mysql_ifaces修改成允許遠端訪問
# mysql_ifaces="127.0.0.1:6032;/tmp/proxysql_admin.sock"
mysql_ifaces="0.0.0.0:6032"
如果ip配置為0.0.0.0表示不限制ip，但是出於安全考慮，admin使用者無論怎麼設定都只能在本機登入，要想遠端登入請看下一小節。
admin_credentials
這個key儲存所有可以操作proxysql的使用者名稱和密碼，格式為：user:pass;user1:pass1，這裡可以修改密碼或定義一個非admin的使用者用於遠端登入。
首先保證想要管理proxysql的機器安裝有mysql client：
-- 先在本機登入
-- 本機IP為192.168.0.201
mysql -h 192.168.0.201 -uadmin -padmin -P6032
([email protected]:p) [d]> update global_variables set variable_value = 'admin:admin;radmin:radmin' where variable_name = 'admin-admin_credentials';
Query OK, 1 row affected (0.00 sec)
([email protected]:p) [d]> LOAD ADMIN VARIABLES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
([email protected]:p) [d]> SAVE ADMIN VARIABLES TO DISK;
Query OK, 31 rows affected (0.00 sec)
([email protected]:p) [d]>
這樣就可以使用下面的命令在其他機器上使用radmin使用者登入（其他機器上也需要有mysql client）：
mysql -uradmin -pradmin -P6032 -h192.168.0.201
mysql>
庫、表說明
mysql> show databases;
+-----+---------------+-------------------------------------+
| seq | name | file |
+-----+---------------+-------------------------------------+
| 0 | main | |
| 2 | disk | /var/lib/proxysql/proxysql.db |
| 3 | stats | |
| 4 | monitor | |
| 5 | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.00 sec)
main 記憶體配置資料庫，表裡存放後端db例項、使用者驗證、路由規則等資訊。表名以runtime_開頭的表示proxysql當前執行的配置內容，不能通過dml語句修改，只能修改對應的不以 runtime_開頭的（在記憶體）裡的表，然後LOAD 使其生效， SAVE 使其存到硬碟以供下次重啟載入。
disk 是持久化到硬碟的配置，sqlite資料檔案。
stats 是proxysql執行抓取的統計資訊，包括到後端各命令的執行次數、流量、processlist、查詢種類彙總/執行時間等等。
monitor 庫儲存 monitor模組收集的資訊，主要是對後端db的健康/延遲檢查。
更多表介紹、配置介紹：MySQL ProxySQL讀寫分離使用初探
負載均衡
在proxysql中新增資料庫server資訊
-- 叢集中有三個節點
192.168.0.101:3306
192.168.0.102:3306
192.168.0.103:3306
-- proxysql安裝在
192.168.0.201:6032
-- 登入到192.168.0.201，使用遠端連線使用者radmin登入
# mysql -h192.168.0.201 -P6032 -uradmin -pradmin
mysql> use main;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+--------------------------------------------+
| tables |
+--------------------------------------------+
| global_variables |
| mysql_collations |
| mysql_group_replication_hostgroups |
| mysql_query_rules |
| mysql_query_rules_fast_routing |
| mysql_replication_hostgroups |
| mysql_servers |
| mysql_users |
| proxysql_servers |
| runtime_checksums_values |
| runtime_global_variables |
| runtime_mysql_group_replication_hostgroups |
| runtime_mysql_query_rules |
| runtime_mysql_query_rules_fast_routing |
| runtime_mysql_replication_hostgroups |
| runtime_mysql_servers |
| runtime_mysql_users |
| runtime_proxysql_servers |
| runtime_scheduler |
| scheduler |
+--------------------------------------------+
20 rows in set (0.00 sec)
-- 這裡用到mysql_servers
insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(1,'192.168.0.101','3306',1,'Write Group');
insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(1,'192.168.0.102','3306',2,'Read Group');
insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(1,'192.168.0.103','3306',2,'Read Group');
-- 特別注意這裡兩條資料的hostgroup_id是一樣的，權重weight也是一樣的，前者保證同一個group以便負載均衡，後者權重可以影響負載均衡結果
mysql> select hostgroup_id,hostname,weight from mysql_servers;
+--------------+---------------+--------+
| hostgroup_id | hostname | weight |
+--------------+---------------+--------+
| 1 | 192.168.0.101 | 1 |
| 1 | 192.168.0.102 | 1 |
| 1 | 192.168.0.103 | 1 |
+--------------+---------------+--------+
3 rows in set (0.00 sec)
在資料庫server中新增賬號
首先在proxysql中確認監控使用者名稱和密碼
mysql> select * from global_variables where variable_name in ('mysql-monitor_username','mysql-monitor_password');
+------------------------+----------------+
| variable_name | variable_value |
+------------------------+----------------+
| mysql-monitor_password | monitor |
| mysql-monitor_username | monitor |
+------------------------+----------------+
2 rows in set (0.01 sec)
然後在所有server中新增賬號：監控賬號和程式賬號，其中監控賬號只需要USAGE許可權；程式賬號是開放給使用者使用的，需要業務要求的許可權。兩者的使用者名稱和密碼都可以任意修改，但是配置的地方不一樣，前者在global_variables表，後者在mysql_users表。
在任意一個節點操作即可，因為是galera叢集，其他節點會自動同步：
# /mysql/app/proxysql_galera/program/bin/mysql -uroot -p1234 -S /mysql/app/proxysql_galera/program/proxysql_galera.sock
-- 監控賬戶
mysql> CREATE USER 'monitor'@'%' IDENTIFIED BY 'monitor';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT USAGE ON *.* TO 'monitor'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
-- 程式賬戶
mysql> CREATE USER 'proxysql'@'%' IDENTIFIED BY 'proxysql';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL ON *.* TO 'proxysql'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT user,host FROM mysql.user;
+---------------+-----------+
| user | host |
+---------------+-----------+
| monitor | % |
| proxysql | % |
| root | % |
| mysql.session | localhost |
| mysql.sys | localhost |
+---------------+-----------+
5 rows in set (0.00 sec)
在proxysql中新增程式賬號資訊
# mysql -h192.168.0.201 -P6032 -uradmin -pradmin
mysql> use main;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
-- 這裡需要注意，default_hostgroup需要和上面的mysql_servers.hostgroup_id對應
INSERT INTO mysql_users(username,password,default_hostgroup) VALUES ('proxysql','proxysql',1);
使配置生效
因為修改了mysql_users和mysql_servers(還可能修改了global_variables)，這三個表在上面的配置小節中可以找到是屬於Memory級別的，如果需要立即生效，需要載入到Runtime；同時應該持久化到Disk。
# mysql -h192.168.0.201 -P6032 -uradmin -pradmin
mysql> use main;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
-- Memory -> Runtime
mysql> LOAD MYSQL SERVERS TO RUNTIME;
Query OK, 0 rows affected (0.01 sec)
mysql> LOAD MYSQL USERS TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
mysql> LOAD MYSQL VARIABLES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
-- Memory -> Disk
mysql> SAVE MYSQL SERVERS TO DISK;
Query OK, 0 rows affected (0.02 sec)
mysql> SAVE MYSQL USERS TO DISK;
Query OK, 0 rows affected (0.01 sec)
mysql> SAVE MYSQL VARIABLES TO DISK;
Query OK, 94 rows affected (0.00 sec)
驗證負載均衡
proxysql的6032埠是管理入口，6033埠就是客戶端入口。
先寫一個sql指令碼查詢一個可以區分到底呼叫到哪個主機的引數：
vi test_proxysql_lb.sql
select @@server_id;
ESC
:wq
再寫一個shell指令碼迴圈200次呼叫這個sql語句並把檔案輸出到/tmp/test_proxysql_lb.txt:
#!/bin/bash
i=0
while(($i<200))
do
/mysql/app/test/program/bin/mysql -h192.168.0.201 -P6033 -uproxysql -pproxysql < test_proxysql_lb.sql >> /tmp/test_proxy_sql_lb.txt
let "i++"
echo "$i"
sleep 0.1
done
執行後檢查結果：
# grep -nc '118' /tmp/test_proxy_sql_lb.txt
103
# grep -nc '119' /tmp/test_proxy_sql_lb.txt
97
結果符合預期。
負載均衡過程解析
首先，proxysql接收到來自6033埠proxysql使用者的請求：mysql -h192.168.0.201 -P6033 -uproxysql -pproxysql < test_proxysql_lb.sql
因為mysql_user表username + frontend是唯一鍵，proxysql查到這個使用者繫結的hostgroup_id為100，因此會被應用到192.168.11.118或192.168.11.119上
根據server的權重來做負載均衡，分配到對應的主機。
這裡沒有配置mysql_query_rules，預設使用mysql_users中使用者的default_hostgroup，可以在mysql_query_rules中新增路由規則，讓匹配到規則的使用者、sql、代理埠等等路由到特定的組。
讀寫分離
基於上面的負載均衡，一起配置負載均衡和讀寫分離，分配以下主機：
-- 一主多從master
192.168.0.101:3306
-- slaves
192.168.0.102:3306
192.168.0.103:3306
在proxysql中新增資料庫server資訊
-- 登入到192.168.0.201，使用遠端連線使用者radmin登入
# mysql -h192.168.0.201 -P6032 -uradmin -pradmin
mysql> use main;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
-- 插入server資訊，其中master和slave的hostgroup_id要不一樣
insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(1001,'192.168.0.101','3306',1,'Write Group');
insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(1002,'192.168.0.102','3306',2,'Read Group');
insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(1002,'192.168.0.103','3306',2,'Read Group');
Query OK, 3 rows affected (0.00 sec)
mysql> select hostgroup_id,hostname,weight from mysql_servers where hostgroup_id >= 1001;
-- 這裡準備把主庫也配置為可讀，配置較低的權重
+--------------+---------------+--------+
| hostgroup_id | hostname | weight |
+--------------+---------------+--------+
| 1001 | 192.168.0.101 | 1 |
| 1002 | 192.168.0.101 | 1 |
| 1002 | 192.168.0.102 | 2 |
| 1002 | 192.168.0.103 | 2 |
+--------------+---------------+--------+
4 rows in set (0.00 sec)
在資料庫server中新增賬號
首先在proxysql中確認監控使用者名稱和密碼
mysql> select * from global_variables where variable_name in ('mysql-monitor_username','mysql-monitor_password');
+------------------------+----------------+
| variable_name | variable_value |
+------------------------+----------------+
| mysql-monitor_password | monitor |
| mysql-monitor_username | monitor |
+------------------------+----------------+
2 rows in set (0.01 sec)
然後在所有server中新增賬號：監控賬號和程式賬號，其中監控賬號只需要USAGE許可權；程式賬號是開放給使用者使用的，需要業務要求的許可權。兩者的使用者名稱和密碼都可以任意修改，但是配置的地方不一樣，前者在global_variables表，後者在mysql_users表。
在master節點操作即可，因為是mgr單主叢集，其他節點會自動同步：
# /mysql/app/proxysql_ms_test/program/bin/mysql -uroot -p1234 -S /mysql/app/proxysql_ms_test/program/proxysql_ms_test.sock
-- 監控賬戶
mysql> CREATE USER 'monitor'@'%' IDENTIFIED BY 'monitor';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT USAGE ON *.* TO 'monitor'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
-- 程式賬戶，擁有所有業務許可權（這裡給ALL）
mysql> CREATE USER 'proxysql_msRW'@'%' IDENTIFIED BY 'proxysql_msRW';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL ON *.* TO 'proxysql_msRW'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT user,host FROM mysql.user;
+---------------+-----------+
| user | host |
+---------------+-----------+
| monitor | % |
| proxysql_msRW | % |
| root | % |
| rpuser | % |
| mysql.session | localhost |
| mysql.sys | localhost |
+---------------+-----------+
7 rows in set (0.01 sec)
在proxysql中新增程式賬號資訊
# mysql -h192.168.0.201 -P6032 -uradmin -pradmin
mysql> use main;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
-- 這裡需要注意，default_hostgroup需要和上面的mysql_servers.hostgroup_id對應，master使用RW賬戶，slave使用R賬戶
INSERT INTO mysql_users(username,password,default_hostgroup) VALUES ('proxysql_msRW', 'proxysql_msRW', 1001);
在proxysql中新增路由規則
讀寫規則：以select開頭的且不以for update結尾的語句一律路由到slave節點，其他的路由到master節點。
INSERT INTO mysql_query_rules(active,username,match_pattern,destination_hostgroup,apply) VALUES(1,'proxysql_msRW','^SELECT.*FOR UPDATE$',1001,1),(1,'proxysql_msRW','^SELECT',1002,1);
Query OK, 2 rows affected (0.00 sec)
mysql> select destination_hostgroup hostgroup,username,match_patternfrom mysql_query_rules where destination_hostgroup >= 1001;
+-----------+---------------+----------------------+
| hostgroup | username | match_pattern |
+-----------+---------------+----------------------+
| 1001 | proxysql_msRW | ^SELECT.*FOR UPDATE$ |
| 1002 | proxysql_msRW | ^SELECT |
+-----------+---------------+----------------------+
2 rows in set (0.00 sec)
使配置生效
因為修改了mysql_users和mysql_servers(還可能修改了global_variables)，這三個表在上面的配置小節中可以找到是屬於Memory級別的，如果需要立即生效，需要載入到Runtime；同時應該持久化到Disk。
-- Memory -> Runtime
mysql> LOAD MYSQL SERVERS TO RUNTIME;
Query OK, 0 rows affected (0.01 sec)
mysql> LOAD MYSQL USERS TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
mysql> LOAD MYSQL VARIABLES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
-- Memory -> Disk
mysql> SAVE MYSQL SERVERS TO DISK;
Query OK, 0 rows affected (0.02 sec)
mysql> SAVE MYSQL USERS TO DISK;
Query OK, 0 rows affected (0.01 sec)
mysql> SAVE MYSQL VARIABLES TO DISK;
Query OK, 94 rows affected (0.00 sec)
驗證讀寫分離
先驗證負載均衡是否是正常的：
#!/bin/bash
i=0
while(($i<500))
do
/mysql/app/test/program/bin/mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e 'select @@relay_log_basename' >> /tmp/test_proxy_sql_lb.txt
let "i++"
echo "$i"
sleep 0.1
done
和負載均衡一樣執行後，統計行數：
#11.119
grep -nc 'mysql119' /tmp/test_proxy_sql_lb.txt
176
#20.8
grep -nc 'qwer1' /tmp/test_proxy_sql_lb.txt
223
#20.7
grep -nc 's011' /tmp/test_proxy_sql_lb.txt
101
大致是2:2:1，主節點的查詢是從節點的1/2。
上面的查詢可以用來驗證select是否被正確地路由，在proxysql上：
mysql> select hostgroup,username,digest_text,count_star from stats_mysql_query_digest where hostgroup>1000;
+-----------+---------------+-----------------------------+------------+
| hostgroup | username | digest_text | count_star |
+-----------+---------------+-----------------------------+------------+
| 1002 | proxysql_msRW | select @@relay_log_basename | 1212 |
+-----------+---------------+-----------------------------+------------+
1 row in set (0.00 sec)
可以看到所有select都被路由到了1002(slave+master)
然後驗證讀寫分離
mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e 'drop schema if exists test';
mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "create schema if not exists test_rw";
mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "create table if not exists test_rw.test_rw(id int(11) not null,primary key(id)) engine=innodb charset=utf8 comment '測試讀寫分離'";
mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "insert into test_rw.test_rw(id) values (1)";
mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "insert into test_rw.test_rw(id) values (2)";
...
/mysql/app/xftest/program/bin/mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "insert into test_rw.test_rw(id) values (6)";
/mysql/app/xftest/program/bin/mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "delete from test_rw.test_rw";
-- 新增500條資料
#!/bin/bash
i=0
while(($i<500))
do
/mysql/app/xftest/program/bin/mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "insert into test_rw.test_rw(id) values ($i)" >> /tmp/test_proxy_sql_lb.txt
let "i++"
echo "$i"
sleep 0.1
done
-- 查詢500次
#!/bin/bash
i=0
while(($i<500))
do
/mysql/app/xftest/program/bin/mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "select * from test_rw.test_rw" >> /tmp/test_proxy_sql_lb.txt
let "i++"
echo "$i"
sleep 0.1
done
-- select for update 500次
#!/bin/bash
i=0
while(($i<500))
do
/mysql/app/xftest/program/bin/mysql -h192.168.0.201 -P6033 -uproxysql_msRW -pproxysql_msRW -e "select * from test_rw.test_rw limit 1 for update;commit" >> /tmp/test_proxy_sql_lb.txt
let "i++"
echo "$i"
sleep 0.1
done
在proxysql查詢統計和命中次數：
mysql> select hostgroup,username,digest_text,count_star from stats_mysql_query_digest where hostgroup > 1000;
+-----------+--------------------------------------------------+------------+
| hostgroup | digest_text | count_star |
+-----------+--------------------------------------------------+------------+
| 1001 | insert into test_rw.test_rw(id) values (?) | 506 |
| 1001 | create table test_rw.test_rw(id int(?) ... | 1 |
| 1001 | create schema test_rw | 1 |
| 1001 | delete from test_rw.test_rw | 1 |
| 1001 | select * from test_rw.test_rw limit ? for update | 502 |
| 1001 | drop schema test | 1 |
| 1001 | commit | 502 |
| 1002 | select * from test_rw.test_rw | 500 |
+-----------+--------------------------------------------------+------------+
都正確分配到了對應的主機上。
proxysql分庫(same MySQL Server different schemas)
分庫和讀寫分離基本類似，都是配置規則，路由到不同的機器。
分庫可以分為同例項不同庫、不同例項相同庫、不同例項不同庫，這裡暫時只研究同例項不同庫，其它種情況可以參考這裡：MySQL Sharding with ProxySQL
模擬分庫
場景：假設某一個業務流程需要記錄流水記錄，每天的流水非常多，如果放在同一張表，很快就會爆炸，所以按照月份分庫，同一個月的資料放在一張表裡。
依然使用負載均衡小節的galera：
mysql> create schema user_201804;
Query OK, 1 row affected (0.00 sec)
mysql> create schema user_201803;
Query OK, 1 row affected (0.00 sec)
mysql> create schema user_201802;
Query OK, 1 row affected (0.00 sec)
mysql> create schema user_201801;
Query OK, 1 row affected (0.00 sec)
mysql> use user_201801;
Database changed
mysql> CREATE TABLE `user_op_history` (
-> `id` INT (20) NOT NULL AUTO_INCREMENT,
-> `user_id` INT (20) NOT NULL COMMENT "使用者ID",
-> `operate_type` VARCHAR (64) NOT NULL COMMENT '操作型別',
-> `operate_time` datetime NOT NULL COMMENT '操作時間',
-> PRIMARY KEY (`id`)
-> ) ENGINE = INNODB CHARSET = UTF8 COMMENT '使用者操作流水錶';
Query OK, 0 rows affected (0.01 sec)
mysql> INSERT INTO user_op_history(user_id,operate_type,operate_time) VALUES (65535,"login","2018-01-01 00:00:01"),(65535,"pay","2018-01-01 00:00:10"),(65535,"logout","2018-01-01 00:01:01");
Query OK, 3 rows affected (0.00 sec)
Records: 3 Duplicates: 0 Warnings: 0
-- 其他三個庫做類似操作
配置查詢規則
查詢時在sql裡指定年月(YYYYMM)，proxysql根據年月對映到對應的庫上執行sql，比如查詢201801的資料規定這樣查詢：
select /* month=201801 */ * from user.user_op_history;
1
查詢規則首先匹配/* month=201801 */，遇到這樣註釋，表明查詢時需要分庫，進入規則鏈；然後一級一級替換或者迴圈替換user.為user_201801.。
proxy_sql規則是可以迴圈呼叫自己做替換的，關鍵是mysql_query_rules表的三個欄位：apply,flagIN,flagOUT，原理如下：
1. 一個sql查詢通過proxysql時，proxysql首先找flagIN=0的規則進行匹配，如果沒有找到，就直接使用原sql到目標庫查詢；
2. 匹配到後，做相應規則替換，如果規則的apply=1，那麼本次替換後的sql將被放到目標庫執行；
3. 如果apply=0且flagOUT!=0，表示替換後進入下一個flagIN=X的規則，如果匹配到多條，則選擇第一個找到的規則；
4. 繼續3中的步驟，直到apply=1或flagOUT=0或者規則鏈迭代次數超過mysql-query_processor_iterations定義的最大次數；
5. 應用最終的sql到目標庫。
首先配置mysql-query_processor_iterations=10：
# mysql -h192.168.0.201 -P6032 -uradmin -pradmin
mysql> use main;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> update global_variables set variable_value=10 where variable_name='mysql-query_processor_iterations';
mysql> LOAD MYSQL VARIABLES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
mysql> SAVE MYSQL VARIABLES TO DISK;
Query OK, 94 rows affected (0.00 sec)
然後插入查詢規則：
mysql> INSERT INTO mysql_query_rules (rule_id,active,username,match_pattern,replace_pattern,apply,FlagOUT,FlagIN) VALUES (101,1,'proxysql',"\S*\s*\/\*\s*month=(\d+)\s*\*.*",null,0,1001,0),(102,1,'proxysql','(\S*\s*\/\*\s*month=(\d+)\s*\*.*)user\.(.*)','\1user_\2.\3',0,1001,1001);
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
mysql> SAVE MYSQL QUERY RULES TO DISK;
Query OK, 0 rows affected (0.00 sec)
mysql> select match_pattern,replace_pattern,apply,flagIN,flagOUT from mysql_query_rules;
+---------------------------------------------+-----------------+-------+--------+---------+
| match_pattern | replace_pattern | apply | flagIN | flagOUT |
+---------------------------------------------+-----------------+-------+--------+---------+
| \S*\s*\/\*\s*month=(\d+)\s*\*.* | NULL | 0 | 0 | 1001 |
| (\S*\s*\/\*\s*month=(\d+)\s*\*.*)user\.(.*) | \1user_\2.\3 | 0 | 1001 | 1001 |
+---------------------------------------------+-----------------+-------+--------+---------+
插入的兩條規則，上面一條flagIN=0，匹配到/* month=(\d+) */將進入規則鏈，不做任何替換replace_pattern=NULL；
然後進入flagIN=1001的規則，替換user.為user_${month}.；
接著繼續進入flagIN=1001的規則，迴圈替換，直到匹配失敗，或者超過迭代次數限制；
測試
-- 這裡特別注意，在command line執行sql一定要帶上--comments引數，否則會skip comments
# /mysql/app/test/program/bin/mysql --comments -h192.168.0.201 -P6033 -uproxysql -pproxysql
mysql> select /* month=201801 */ * from user.user_op_history;
+----+---------+--------------+---------------------+
| id | user_id | operate_type | operate_time |
+----+---------+--------------+---------------------+
| 1 | 65535 | login | 2018-01-01 00:00:01 |
| 3 | 65535 | pay | 2018-01-01 00:00:10 |
| 5 | 65535 | logout | 2018-01-01 00:01:01 |
+----+---------+--------------+---------------------+
3 rows in set (0.00 sec)
mysql> select /* month=201802 */ a.operate_time,b.operate_time from user.user_op_history a left join user.user_op_history b ON 1=1;
+---------------------+---------------------+
| operate_time | operate_time |
+---------------------+---------------------+
| 2018-02-05 11:10:01 | 2018-02-05 11:10:01 |
| 2018-02-10 00:00:10 | 2018-02-05 11:10:01 |
| 2018-02-11 12:01:01 | 2018-02-05 11:10:01 |
| 2018-02-05 11:10:01 | 2018-02-10 00:00:10 |
| 2018-02-10 00:00:10 | 2018-02-10 00:00:10 |
| 2018-02-11 12:01:01 | 2018-02-10 00:00:10 |
| 2018-02-05 11:10:01 | 2018-02-11 12:01:01 |
| 2018-02-10 00:00:10 | 2018-02-11 12:01:01 |
| 2018-02-11 12:01:01 | 2018-02-11 12:01:01 |
+---------------------+---------------------+
9 rows in set (0.00 sec)
mysql> insert into /* month=201802 */ user.user_op_history(user_id,operate_type,operate_time) values (95553,"post","2018-02-28 11:11:11");
Query OK, 1 row affected (0.00 sec)
mysql> select * /* month=201802 */ from user.user_op_history;
+----+---------+--------------+---------------------+
| id | user_id | operate_type | operate_time |
+----+---------+--------------+---------------------+
| 1 | 95553 | login | 2018-02-05 11:10:01 |
| 3 | 65535 | pay | 2018-02-10 00:00:10 |
| 5 | 95553 | comment | 2018-02-11 12:01:01 |
| 7 | 95553 | post | 2018-02-28 11:11:11 |
+----+---------+--------------+---------------------+
4 rows in set (0.00 sec)
mysql> update /* month=201802 */ user.user_op_history set operate_time="2018-02-28 12:12:12" where id=7;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select * /* month=201802 */ from user.user_op_history;
+----+---------+--------------+---------------------+
| id | user_id | operate_type | operate_time |
+----+---------+--------------+---------------------+
| 1 | 95553 | login | 2018-02-05 11:10:01 |
| 3 | 65535 | pay | 2018-02-10 00:00:10 |
| 5 | 95553 | comment | 2018-02-11 12:01:01 |
| 7 | 95553 | post | 2018-02-28 12:12:12 |
+----+---------+--------------+---------------------+
4 rows in set (0.00 sec)
JDBC連線proxysql
package com.enmo.dbaas;
import java.sql.*;
import java.util.Scanner;
/**
* Hello world!
*/
public class App {
private final static String JDBC_URL = "jdbc:mysql://192.168.0.201:6033";
private final static String USERNAME = "proxysql";
private final static String PASSWORD = "proxysql";
static {
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
public static void main(String[] args) throws SQLException {
Connection conn = DriverManager.getConnection(JDBC_URL, USERNAME, PASSWORD);
Statement st = conn.createStatement();
String sql = null;
do {
sql = new Scanner(System.in).nextLine();
if (sql != null && !sql.trim().isEmpty()) {
sql = sql.trim();
System.out.println(sql);
if (sql.startsWith("select")) {
ResultSet resultSet = st.executeQuery(sql);
System.out.println("id\tuser_id\toperate_type\toperate_time");
while (resultSet.next()) {
System.out.println(
resultSet.getString("id") +
"\t" +
resultSet.getString("user_id") +
"\t" +
resultSet.getString("operate_type") +
"\t" +
resultSet.getString("operate_time")
);
}
} else if (sql.startsWith("update")) {
Integer rows = st.executeUpdate(sql);
System.out.println("Query OK, " + rows + " row affected");
} else {
System.out.println("Query " + (st.execute(sql) ? "OK" : "FAILED"));
}
}
} while (sql != null && !sql.trim().isEmpty());
System.out.println("Bye!");
}
}
開啟WEB統計
首先開啟web功能
mysql> update global_variables set variable_value='true' where variable_name='admin-web_enabled';
Query OK, 1 row affected (0.00 sec)
mysql> LOAD ADMIN VARIABLES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
mysql> SAVE ADMIN VARIABLES TO DISK;
Query OK, 31 rows affected (0.00 sec)
然後檢視埠和登入web介面的使用者名稱和密碼，使用者名稱和密碼與stat賬戶一致：
mysql> select * from global_variables where variable_name LIKE 'admin-web%' or variable_name LIKE 'admin-stats%';
+-----------------------------------+----------------+
| variable_name | variable_value |
+-----------------------------------+----------------+
| admin-stats_credentials | stats:stats |<--賬戶密碼
| admin-stats_mysql_connections | 60 |
| admin-stats_mysql_connection_pool | 60 |
| admin-stats_mysql_query_cache | 60 |
| admin-stats_system_cpu | 60 |
| admin-stats_system_memory | 60 |
| admin-web_enabled | true |
| admin-web_port | 6080 |<--埠
+-----------------------------------+----------------+
8 rows in set (0.00 sec)
訪問192.168.0.201:6080並使用stats:stats登入即可檢視一些統計資訊。
scheduler列印狀態到日誌
先定義一個指令碼，往日誌裡面寫資料：
vi /log/app/dbaas/proxysql/status.sh
#!/bin/bash
DATE=`date "+%Y-%m-%d %H:%M:%S"`
echo "{\"dateTime\":\"$DATE\",\"status\":\"running\"}">> /log/app/dbaas/proxysql/status.log
ESC
:wq
chmod 777 /log/app/dbaas/proxysql/status.sh
然後在proxysql插入一條scheduler：
mysql> insert into scheduler(active,interval_ms,filename) values (1,60000,'/log/app/dbaas/proxysql/status.sh');
Query OK, 1 row affected (0.00 sec)
mysql> LOAD SCHEDULER TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)
mysql> SAVE SCHEDULER TO DISK;
Query OK, 0 rows affected (0.01 sec)
檢視日誌就可以看到結果了：
{"dateTime":"2018-04-20 16:12:32","status":"running"}
{"dateTime":"2018-04-20 16:13:32","status":"running"}
{"dateTime":"2018-04-20 16:14:32","status":"running"}
{"dateTime":"2018-04-20 16:15:32","status":"running"}
{"dateTime":"2018-04-20 16:16:32","status":"running"}
{"dateTime":"2018-04-20 16:17:32","status":"running"}
{"dateTime":"2018-04-20 16:18:32","status":"running"}
{"dateTime":"2018-04-20 16:19:32","status":"running"}
{"dateTime":"2018-04-20 16:20:32","status":"running"}
叢集
proxysql可以配置叢集實現高可用ProxySQL Cluster。
暫不研究。
MHA結合ProxySQL
insert into mysql_replication_hostgroups (writer_hostgroup,reader_hostgroup,comment)values(1001,1002,'測試讀寫分離高可用');
使用 sysbench 做壓測
INSERT INTO mysql_users(username,password,default_hostgroup) VALUES ('sbuser','sbpass',1001);
sysbench --test=oltp --oltp-table-size=4000 --oltp-read-only=off --init-rng=on --num-threads=5 --max-requests=0 --oltp-dist-type=uniform --max-time=36 --mysql-user=sbuser --mysql-password='sbpass' --db-driver=mysql --mysql-port=6033 --mysql-table-engine=innodb --mysql-host=127.0.0.1 --mysql-db=sbtest --oltp-tables-count=5 --report-interval=1 prepare
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
WARNING: --max-time is deprecated, use --time instead
sysbench 1.0.9 (using system LuaJIT 2.0.4)
Creating table 'sbtest1'...
Inserting 4000 records into 'sbtest1'
Creating secondary indexes on 'sbtest1'...
Creating table 'sbtest2'...
Inserting 4000 records into 'sbtest2'
Creating secondary indexes on 'sbtest2'...
Creating table 'sbtest3'...
Inserting 4000 records into 'sbtest3'
Creating secondary indexes on 'sbtest3'...
Creating table 'sbtest4'...
Inserting 4000 records into 'sbtest4'
Creating secondary indexes on 'sbtest4'...
Creating table 'sbtest5'...
Inserting 4000 records into 'sbtest5'
Creating secondary indexes on 'sbtest5'...
sysbench --test=oltp --oltp-table-size=4000 --oltp-read-only=off --init-rng=on --num-threads=5 --max-requests=0 --oltp-dist-type=uniform --max-time=36 --mysql-user=sbuser --mysql-password='sbpass' --db-driver=mysql --mysql-port=6033 --mysql-table-engine=innodb --mysql-host=127.0.0.1 --mysql-db=sbtest --oltp-tables-count=5 --report-interval=1 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
WARNING: --max-time is deprecated, use --time instead
sysbench 1.0.9 (using system LuaJIT 2.0.4)
Running the test with following options:
Number of threads: 5
Report intermediate results every 1 second(s)
Initializing random number generator from current time
Initializing worker threads...
Threads started!
[ 1s ] thds: 5 tps: 88.01 qps: 1817.17 (r/w/o: 1276.56/359.76/180.85) lat (ms,95%): 155.80 err/s: 0.97 reconn/s: 0.00
[ 2s ] thds: 5 tps: 22.64 qps: 507.26 (r/w/o: 356.01/104.95/46.30) lat (ms,95%): 193.38 err/s: 0.00 reconn/s: 0.00
[ 3s ] thds: 5 tps: 159.66 qps: 3166.04 (r/w/o: 2220.15/626.58/319.32) lat (ms,95%): 82.96 err/s: 0.00 reconn/s: 0.00
[ 4s ] thds: 5 tps: 176.95 qps: 3555.91 (r/w/o: 2493.24/707.78/354.89) lat (ms,95%): 52.89 err/s: 1.00 reconn/s: 0.00
[ 5s ] thds: 5 tps: 175.05 qps: 3503.02 (r/w/o: 2448.72/704.21/350.10) lat (ms,95%): 69.29 err/s: 0.00 reconn/s: 0.00
[ 6s ] thds: 5 tps: 182.01 qps: 3612.16 (r/w/o: 2528.11/720.03/364.02) lat (ms,95%): 50.11 err/s: 0.00 reconn/s: 0.00
[ 7s ] thds: 5 tps: 179.00 qps: 3598.93 (r/w/o: 2516.95/723.99/357.99) lat (ms,95%): 43.39 err/s: 0.00 reconn/s: 0.00
[ 8s ] thds: 5 tps: 192.96 qps: 3835.11 (r/w/o: 2683.38/767.82/383.91) lat (ms,95%): 41.85 err/s: 0.00 reconn/s: 0.00
[ 9s ] thds: 5 tps: 164.96 qps: 3336.26 (r/w/o: 2341.48/662.85/331.93) lat (ms,95%): 66.84 err/s: 0.00 reconn/s: 0.00
[ 10s ] thds: 5 tps: 197.88 qps: 3917.56 (r/w/o: 2741.29/780.51/395.75) lat (ms,95%): 44.17 err/s: 0.00 reconn/s: 0.00
[ 11s ] thds: 5 tps: 181.20 qps: 3710.20 (r/w/o: 2603.94/741.84/364.41) lat (ms,95%): 46.63 err/s: 2.00 reconn/s: 0.00
[ 12s ] thds: 5 tps: 183.96 qps: 3617.31 (r/w/o: 2528.52/720.86/367.93) lat (ms,95%): 53.85 err/s: 0.00 reconn/s: 0.00
[ 13s ] thds: 5 tps: 175.97 qps: 3502.33 (r/w/o: 2446.53/703.86/351.93) lat (ms,95%): 49.21 err/s: 0.00 reconn/s: 0.00
[ 14s ] thds: 5 tps: 169.03 qps: 3481.62 (r/w/o: 2444.43/698.12/339.06) lat (ms,95%): 64.47 err/s: 1.00 reconn/s: 0.00
[ 15s ] thds: 5 tps: 186.00 qps: 3661.99 (r/w/o: 2565.99/724.00/372.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 16s ] thds: 5 tps: 177.01 qps: 3572.15 (r/w/o: 2505.11/712.03/355.02) lat (ms,95%): 62.19 err/s: 1.00 reconn/s: 0.00
[ 17s ] thds: 5 tps: 196.95 qps: 3975.97 (r/w/o: 2782.28/799.79/393.90) lat (ms,95%): 50.11 err/s: 0.00 reconn/s: 0.00
[ 18s ] thds: 5 tps: 203.03 qps: 4058.60 (r/w/o: 2842.42/809.12/407.06) lat (ms,95%): 48.34 err/s: 1.00 reconn/s: 0.00
[ 19s ] thds: 5 tps: 136.98 qps: 2739.53 (r/w/o: 1917.67/548.91/272.95) lat (ms,95%): 101.13 err/s: 1.00 reconn/s: 0.00
[ 20s ] thds: 5 tps: 186.04 qps: 3755.77 (r/w/o: 2632.54/749.15/374.08) lat (ms,95%): 47.47 err/s: 0.00 reconn/s: 0.00
[ 21s ] thds: 5 tps: 190.04 qps: 3803.73 (r/w/o: 2660.51/763.15/380.07) lat (ms,95%): 41.85 err/s: 0.00 reconn/s: 0.00
[ 22s ] thds: 5 tps: 190.87 qps: 3797.44 (r/w/o: 2658.21/757.49/381.74) lat (ms,95%): 42.61 err/s: 0.00 reconn/s: 0.00
[ 23s ] thds: 5 tps: 192.13 qps: 3834.55 (r/w/o: 2691.79/758.51/384.26) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 24s ] thds: 5 tps: 157.97 qps: 3166.45 (r/w/o: 2209.62/640.89/315.95) lat (ms,95%): 47.47 err/s: 0.00 reconn/s: 0.00
[ 25s ] thds: 5 tps: 183.00 qps: 3693.10 (r/w/o: 2590.07/736.02/367.01) lat (ms,95%): 38.94 err/s: 1.00 reconn/s: 0.00
[ 26s ] thds: 5 tps: 190.93 qps: 3802.67 (r/w/o: 2668.07/752.74/381.87) lat (ms,95%): 41.85 err/s: 0.00 reconn/s: 0.00
[ 27s ] thds: 5 tps: 174.99 qps: 3520.75 (r/w/o: 2466.83/701.95/351.98) lat (ms,95%): 47.47 err/s: 2.00 reconn/s: 0.00
[ 28s ] thds: 5 tps: 180.10 qps: 3607.00 (r/w/o: 2522.40/723.40/361.20) lat (ms,95%): 49.21 err/s: 1.00 reconn/s: 0.00
[ 29s ] thds: 5 tps: 191.94 qps: 3846.83 (r/w/o: 2700.18/761.77/384.88) lat (ms,95%): 36.24 err/s: 1.00 reconn/s: 0.00
[ 30s ] thds: 5 tps: 178.95 qps: 3611.02 (r/w/o: 2529.31/722.80/358.90) lat (ms,95%): 59.99 err/s: 1.00 reconn/s: 0.00
[ 31s ] thds: 5 tps: 195.09 qps: 3859.84 (r/w/o: 2695.28/774.37/390.19) lat (ms,95%): 44.98 err/s: 0.00 reconn/s: 0.00
[ 32s ] thds: 5 tps: 147.88 qps: 3061.57 (r/w/o: 2154.29/609.52/297.76) lat (ms,95%): 118.92 err/s: 3.00 reconn/s: 0.00
[ 33s ] thds: 5 tps: 191.15 qps: 3858.07 (r/w/o: 2704.15/769.61/384.31) lat (ms,95%): 38.25 err/s: 1.00 reconn/s: 0.00
[ 34s ] thds: 5 tps: 186.92 qps: 3723.50 (r/w/o: 2609.95/739.70/373.85) lat (ms,95%): 44.17 err/s: 0.00 reconn/s: 0.00
[ 35s ] thds: 5 tps: 190.03 qps: 3776.52 (r/w/o: 2644.36/752.10/380.05) lat (ms,95%): 41.10 err/s: 0.00 reconn/s: 0.00
[ 36s ] thds: 5 tps: 176.83 qps: 3556.55 (r/w/o: 2484.59/719.30/352.66) lat (ms,95%): 44.17 err/s: 0.00 reconn/s: 0.00
SQL statistics:
queries performed:
read: 87906
write: 25061
other: 12540
total: 125507
transactions: 6261 (173.83 per sec.)
queries: 125507 (3484.59 per sec.)
ignored errors: 18 (0.50 per sec.)
reconnects: 0 (0.00 per sec.)
General statistics:
total time: 36.0163s
total number of events: 6261
Latency (ms):
min: 5.70
avg: 28.75
max: 839.10
95th percentile: 51.94
sum: 180005.46
Threads fairness:
events (avg/stddev): 1252.2000/44.40
execution time (avg/stddev): 36.0011/0.01
[[email protected] oltp_legacy]#
非常不錯的統計資訊，各種命令的聚合統計，方便我們監控資料庫負載趨勢，一眼就可以看出讀多還是寫多。
[[email protected]][(none)]> select * from stats_mysql_commands_counters where Total_cnt;
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
| Command | Total_Time_us | Total_cnt | cnt_100us | cnt_500us | cnt_1ms | cnt_5ms | cnt_10ms | cnt_50ms | cnt_100ms | cnt_500ms | cnt_1s | cnt_5s | cnt_10s | cnt_INFs |
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
| BEGIN | 7973667 | 6279 | 130 | 1276 | 1761 | 3004 | 86 | 22 | 0 | 0 | 0 | 0 | 0 | 0 |
| COMMIT | 79534414 | 6261 | 0 | 0 | 0 | 407 | 2278 | 3525 | 31 | 15 | 5 | 0 | 0 | 0 |
| CREATE_INDEX | 272109 | 5 | 0 | 0 | 0 | 0 | 0 | 3 | 2 | 0 | 0 | 0 | 0 | 0 |
| CREATE_TABLE | 151516 | 5 | 0 | 0 | 0 | 0 | 0 | 4 | 1 | 0 | 0 | 0 | 0 | 0 |
| DELETE | 5222033 | 6266 | 4 | 3897 | 1463 | 738 | 98 | 65 | 0 | 1 | 0 | 0 | 0 | 0 |
| INSERT | 5484312 | 6271 | 22 | 4097 | 1398 | 683 | 43 | 22 | 1 | 4 | 1 | 0 | 0 | 0 |
| SELECT | 58297142 | 87908 | 8427 | 44156 | 23317 | 11076 | 788 | 142 | 0 | 2 | 0 | 0 | 0 | 0 |
| UPDATE | 12196261 | 12552 | 0 | 6609 | 3770 | 1843 | 213 | 111 | 2 | 4 | 0 | 0 | 0 | 0 |
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
8 rows in set (0.00 sec)
[[email protected]][(none)]> select * from stats_mysql_query_digest order by sum_time desc;
+-----------+------------+----------+--------------------+--------------------------------------------------------------------+------------+------------+------------+----------+----------+----------+
| hostgroup | schemaname | username | digest | digest_text | count_star | first_seen | last_seen | sum_time | min_time | max_time |
+-----------+------------+----------+--------------------+--------------------------------------------------------------------+------------+------------+------------+----------+----------+----------+
| 1001 | sbtest | sbuser | 0x695FBF255DBEB0DD | COMMIT | 6261 | 1542266823 | 1542266859 | 79534414 | 1496 | 677436 |
| 1001 | sbtest | sbuser | 0x9D058B6F3BC2F754 | SELECT c FROM sbtest4 WHERE id=? | 12920 | 1542266823 | 1542266859 | 8611177 | 62 | 247702 |
| 1001 | sbtest | sbuser | 0xFAD1519E4760CBDE | BEGIN | 6279 | 1542266823 | 1542266859 | 7973667 | 44 | 24688 |
| 1001 | sbtest | sbuser | 0x9AF59B998A3688ED | SELECT c FROM sbtest2 WHERE id=? | 13020 | 1542266823 | 1542266859 | 7749860 | 58 | 17425 |
| 1001 | sbtest | sbuser | 0xBF001A0C13781C1D | SELECT c FROM sbtest1 WHERE id=? | 12400 | 1542266823 | 1542266859 | 7562027 | 67 | 19407 |
| 1001 | sbtest | sbuser | 0x0250CB4007721D69 | SELECT c FROM sbtest3 WHERE id=? | 12170 | 1542266823 | 1542266859 | 7549452 | 69 | 31838 |
| 1001 | sbtest | sbuser | 0x03744DC190BC72C7 | SELECT c FROM sbtest5 WHERE id=? | 12280 | 1542266823 | 1542266859 | 7518325 | 67 | 25988 |
| 1001 | sbtest | sbuser | 0x52CE762E6D33D57E | UPDATE sbtest4 SET k=k+? WHERE id=? | 1292 | 1542266823 | 1542266859 | 1637059 | 131 | 256380 |
| 1001 | sbtest | sbuser | 0x16ADA60275E5EFAA | UPDATE sbtest2 SET k=k+? WHERE id=? | 1302 | 1542266823 | 1542266859 | 1567520 | 131 | 210837 |
| 1001 | sbtest | sbuser | 0xE5D5BDF004370D6D | UPDATE sbtest3 SET k=k+? WHERE id=? | 1217 | 1542266823 | 1542266859 | 1459861 | 133 | 246168 |
| 1001 | sbtest | sbuser | 0xC2A4F66B0CA11A02 | SELECT c FROM sbtest4 WHERE id BETWEEN ? AND ? | 1292 | 1542266823 | 1542266859 | 1189315 | 145 | 220745 |
| 1001 | sbtest | sbuser | 0x10CAA414CD11445B | UPDATE sbtest4 SET c=? WHERE id=? | 1290 | 1542266823 | 1542266859 | 1172613 | 108 | 50639 |
| 1001 | sbtest | sbuser | 0x283AA9863F85EFC8 | SELECT DISTINCT c FROM sbtest4 WHERE id BETWEEN ? AND ? ORDER BY c | 1292 | 1542266823 | 1542266859 | 1148798 | 226 | 20208 |
| 1001 | sbtest | sbuser | 0x713A0DB06FCE81E2 | DELETE FROM sbtest2 WHERE id=? | 1300 | 1542266823 | 1542266859 | 1128317 | 98 | 25756 |
| 1001 | sbtest | sbuser | 0xC2E5FCE49337737D | UPDATE sbtest2 SET c=? WHERE id=? | 1301 | 1542266823 | 1542266859 | 1127314 | 106 | 31614 |
| 1001 | sbtest | sbuser | 0x61511F703CBA5EA9 | UPDATE sbtest3 SET c=? WHERE id=? | 1216 | 1542266823 | 1542266859 | 1121390 | 108 | 87922 |
| 1001 | sbtest | sbuser | 0x4AC6CC3E8E66E2A5 | SELECT DISTINCT c FROM sbtest2 WHERE id BETWEEN ? AND ? ORDER BY c | 1302 | 1542266823 | 1542266859 | 1121206 | 227 | 16742 |
| 1001 | sbtest | sbuser | 0xC198E52BCCB481C7 | UPDATE sbtest1 SET k=k+? WHERE id=? | 1240 | 1542266823 | 1542266859 | 1099875 | 126 | 19764 |
| 1001 | sbtest | sbuser | 0xE365BEB555319B9E | DELETE FROM sbtest1 WHERE id=? | 1237 | 1542266823 | 1542266859 | 1085251 | 100 | 106851 |
| 1001 | sbtest | sbuser | 0x0D3830CC26B680E5 | SELECT c FROM sbtest4 WHERE id BETWEEN ? AND ? ORDER BY c | 1292 | 1542266823 | 1542266859 | 1078336 | 173 | 15837 |
| 1001 | sbtest | sbuser | 0x44BCB144058686EB | SELECT DISTINCT c FROM sbtest3 WHERE id BETWEEN ? AND ? ORDER BY c | 1217 | 1542266823 | 1542266859 | 1061036 | 224 | 11978 |
| 1001 | sbtest | sbuser | 0xC19480748AE79B4B | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c | 1240 | 1542266823 | 1542266859 | 1052898 | 223 | 15730 |
| 1001 | sbtest | sbuser | 0x29D13DA4BBD11161 | DELETE FROM sbtest3 WHERE id=? | 1213 | 1542266823 | 1542266859 | 1038050 | 101 | 43783 |
| 1001 | sbtest | sbuser | 0x847CD40BA8EA5175 | SELECT DISTINCT c FROM sbtest5 WHERE id BETWEEN ? AND ? ORDER BY c | 1228 | 1542266823 | 1542266859 | 1022440 | 216 | 8728 |
| 1001 | sbtest | sbuser | 0xFB239BC95A23CA36 | UPDATE sbtest1 SET c=? WHERE id=? | 1238 | 1542266823 | 1542266859 | 1015522 | 107 | 28530 |
| 1001 | sbtest | sbuser | 0xB3A675D2CFC75492 | UPDATE sbtest5 SET c=? WHERE id=? | 1228 | 1542266823 | 1542266859 | 1009438 | 108 | 29595 |
| 1001 | sbtest | sbuser | 0xD6E20400D7C3DF2A | DELETE FROM sbtest4 WHERE id=? | 1289 | 1542266823 | 1542266859 | 1008568 | 100 | 20289 |
| 1001 | sbtest | sbuser | 0x4438008ADF85B3AE | SELECT SUM(K) FROM sbtest4 WHERE id BETWEEN ? AND ? | 1292 | 1542266823 | 1542266859 | 989411 | 136 | 15846 |
| 1001 | sbtest | sbuser | 0x3929CFAB1E6B10B2 | UPDATE sbtest5 SET k=k+? WHERE id=? | 1228 | 1542266823 | 1542266859 | 985669 | 129 | 15876 |
| 1001 | sbtest | sbuser | 0xE0E39D08EEF48154 | DELETE FROM sbtest5 WHERE id=? | 1227 | 1542266823 | 1542266859 | 961847 | 102 | 20704 |
| 1001 | sbtest | sbuser | 0xB809E77ADDD9E05B | SELECT c FROM sbtest5 WHERE id BETWEEN ? AND ? ORDER BY c | 1228 | 1542266823 | 1542266859 | 960234 | 170 | 16840 |
| 1001 | sbtest | sbuser | 0x0FEDA5F1D95F2FBA | SELECT c FROM sbtest3 WHERE id BETWEEN ? AND ? ORDER BY c | 1217 | 1542266823 | 1542266859 | 955165 | 173 | 17425 |
| 1001 | sbtest | sbuser | 0x2BD5CA9A9C3B517D | SELECT c FROM sbtest2 WHERE id BETWEEN ? AND ? ORDER BY c | 1302 | 1542266823 | 1542266859 | 940552 | 169 | 15727 |
| 1001 | sbtest | sbuser | 0x9BC3442A424EA0DC | SELECT SUM(K) FROM sbtest2 WHERE id BETWEEN ? AND ? | 1302 | 1542266823 | 1542266859 | 911498 | 127 | 15727 |
| 1001 | sbtest | sbuser | 0xAC80A5EA0101522E | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c | 1240 | 1542266823 | 1542266859 | 907168 | 176 | 10222 |
| 1001 | sbtest | sbuser | 0x8B9F0559BA064E1C | SELECT SUM(K) FROM sbtest1 WHERE id BETWEEN ? AND ? | 1240 | 1542266823 | 1542266859 | 903067 | 137 | 18206 |
| 1001 | sbtest | sbuser | 0x5D10B1F28B54F53C | SELECT SUM(K) FROM sbtest5 WHERE id BETWEEN ? AND ? | 1228 | 1542266823 | 1542266859 | 892770 | 136 | 20193 |
| 1001 | sbtest | sbuser | 0x381AAD21F4326865 | SELECT c FROM sbtest2 WHERE id BETWEEN ? AND ? | 1302 | 1542266823 | 1542266859 | 873450 | 144 | 12705 |
| 1001 | sbtest | sbuser | 0x3138A29396F2C746 | SELECT SUM(K) FROM sbtest3 WHERE id BETWEEN ? AND ? | 1217 | 1542266823 | 1542266859 | 862453 | 139 | 12688 |
| 1001 | sbtest | sbuser | 0x80D0D77C28AAE370 | SELECT c FROM sbtest5 WHERE id BETWEEN ? AND ? | 1228 | 1542266823 | 1542266859 | 821579 | 145 | 20351 |
| 1001 | sbtest | sbuser | 0x4607134A412A3661 | SELECT c FROM sbtest3 WHERE id BETWEEN ? AND ? | 1217 | 1542266823 | 1542266859 | 811445 | 144 | 18657 |
| 1001 | sbtest | sbuser | 0x51D630CD6F907450 | INSERT INTO sbtest4 (id, k, c, pad) VALUES (?, ?, ?, ?) | 1289 | 1542266823 | 1542266859 | 808009 | 97 | 19468 |
| 1001 | sbtest | sbuser | 0x290B92FD743826DA | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ? | 1240 | 1542266823 | 1542266859 | 803480 | 140 | 8472 |
| 1001 | sbtest | sbuser | 0xD284EA74C062DAA9 | INSERT INTO sbtest2 (id, k, c, pad) VALUES (?, ?, ?, ?) | 1297 | 1542266823 | 1542266859 | 787863 | 97 | 17638 |
| 1001 | sbtest | sbuser | 0x1288C33EA439294B | INSERT INTO sbtest5 (id, k, c, pad) VALUES (?, ?, ?, ?) | 1226 | 1542266823 | 1542266859 | 780454 | 100 | 32579 |
| 1001 | sbtest | sbuser | 0xDE31A8F2C92B7524 | INSERT INTO sbtest3 (id, k, c, pad) VALUES (?, ?, ?, ?) | 1213 | 1542266823 | 1542266859 | 763457 | 98 | 15716 |
| 1001 | sbtest | sbuser | 0xE52A0A0210634DAC | INSERT INTO sbtest1 (id, k, c, pad) VALUES (?, ?, ?, ?) | 1236 | 1542266823 | 1542266859 | 726305 | 98 | 14967 |
+-----------+------------+----------+--------------------+--------------------------------------------------------------------+------------+------------+------------+----------+----------+----------+
47 rows in set (0.00 sec)
INSERT INTO mysql_query_rules(active,match_pattern,destination_hostgroup,apply) VALUES(1,'^SELECT.*FOR UPDATE$',1001,1);
INSERT INTO mysql_query_rules(active,match_pattern,destination_hostgroup,apply) VALUES(1,'^SELECT',1002,1);
load mysql query rules to runtime;
save mysql query rules to disk;
select username,active,match_pattern,destination_hostgroup,apply from mysql_query_rules;
select username,active,match_pattern,destination_hostgroup,apply from runtime_mysql_query_rules;
[[email protected]][main]> select * from stats_mysql_query_digest;
+-----------+------------+----------+--------------------+--------------------------------------------------------------------+------------+------------+------------+-----------+----------+----------+
| hostgroup | schemaname | username | digest | digest_text | count_star | first_seen | last_seen | sum_time | min_time | max_time |
+-----------+------------+----------+--------------------+--------------------------------------------------------------------+------------+------------+------------+-----------+----------+----------+
| 1001 | sbtest | sbuser | 0xE52A0A0210634DAC | INSERT INTO sbtest1 (id, k, c, pad) VALUES (?, ?, ?, ?) | 3859 | 1542266823 | 1542267842 | 2461989 | 98 | 115583 |
| 1001 | sbtest | sbuser | 0x4607134A412A3661 | SELECT c FROM sbtest3 WHERE id BETWEEN ? AND ? | 3829 | 1542266823 | 1542267842 | 2496989 | 144 | 18657 |
| 1001 | sbtest | sbuser | 0x80D0D77C28AAE370 | SELECT c FROM sbtest5 WHERE id BETWEEN ? AND ? | 3874 | 1542266823 | 1542267842 | 2526459 | 144 | 28491 |
| 1001 | sbtest | sbuser | 0x0250CB4007721D69 | SELECT c FROM sbtest3 WHERE id=? | 38290 | 1542266823 | 1542267842 | 23314623 | 67 | 31838 |
| 1001 | sbtest | sbuser | 0x51D630CD6F907450 | INSERT INTO sbtest4 (id, k, c, pad) VALUES (?, ?, ?, ?) | 3890 | 1542266823 | 1542267842 | 2477653 | 97 | 19468 |
| 1001 | sbtest | sbuser | 0xD284EA74C062DAA9 | INSERT INTO sbtest2 (id, k, c, pad) VALUES (?, ?, ?, ?) | 3914 | 1542266823 | 1542267842 | 2403735 | 97 | 22581 |
| 1001 | sbtest | sbuser | 0x5D10B1F28B54F53C | SELECT SUM(K) FROM sbtest5 WHERE id BETWEEN ? AND ? | 3874 | 1542266823 | 1542267842 | 2620869 | 132 | 20193 |
| 1001 | sbtest | sbuser | 0xB809E77ADDD9E05B | SELECT c FROM sbtest5 WHERE id BETWEEN ? AND ? ORDER BY c | 3874 | 1542266823 | 1542267842 | 2899483 | 170 | 38522 |
| 1001 | sbtest | sbuser | 0xE365BEB555319B9E | DELETE FROM sbtest1 WHERE id=? | 3866 | 1542266823 | 1542267842 | 3343824 | 100 | 106851 |
| 1001 | sbtest | sbuser | 0x4AC6CC3E8E66E2A5 | SELECT DISTINCT c FROM sbtest2 WHERE id BETWEEN ? AND ? ORDER BY c | 3923 | 1542266823 | 1542267842 | 3496717 | 225 | 71243 |
| 1001 | sbtest | sbuser | 0xD6E20400D7C3DF2A | DELETE FROM sbtest4 WHERE id=? | 3891 | 1542266823 | 1542267842 | 3073992 | 90 | 25195 |
| 1001 | sbtest | sbuser | 0x9BC3442A424EA0DC | SELECT SUM(K) FROM sbtest2 WHERE id BETWEEN ? AND ? | 3923 | 1542266823 | 1542267842 | 2791933 | 127 | 38469 |
| 1001 | sbtest | sbuser | 0xDE31A8F2C92B7524 | INSERT INTO sbtest3 (id, k, c, pad) VALUES (?, ?, ?, ?) | 3817 | 1542266823 | 1542267842 | 2503361 | 98 | 115583 |
| 1001 | sbtest | sbuser | 0xC198E52BCCB481C7 | UPDATE sbtest1 SET k=k+? WHERE id=? | 3873 | 1542266823 | 1542267842 | 3727858 | 126 | 55478 |
| 1001 | sbtest | sbuser | 0x52CE762E6D33D57E | UPDATE sbtest4 SET k=k+? WHERE id=? | 3900 | 1542266823 | 1542267842 | 4190745 | 130 | 256380 |
| 1001 | sbtest | sbuser | 0x2BD5CA9A9C3B517D | SELECT c FROM sbtest2 WHERE id BETWEEN ? AND ? ORDER BY c | 3923 | 1542266823 | 1542267842 | 2972434 | 169 | 21975 |
| 1001 | sbtest | sbuser | 0x29D13DA4BBD11161 | DELETE FROM sbtest3 WHERE id=? | 3821 | 1542266823 | 1542267842 | 3000168 | 101 | 43783 |
| 1001 | sbtest | sbuser | 0x847CD40BA8EA5175 | SELECT DISTINCT c FROM sbtest5 WHERE id BETWEEN ? AND ? ORDER BY c | 3874 | 1542266823 | 1542267842 | 3234729 | 216 | 13458 |
| 1001 | sbtest | sbuser | 0x10CAA414CD11445B | UPDATE sbtest4 SET c=? WHERE id=? | 3893 | 1542266823 | 1542267842 | 3503356 | 108 | 50639 |
| 1001 | sbtest | sbuser | 0xAC80A5EA0101522E | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c | 3873 | 1542266823 | 1542267842 | 2848355 | 172 | 12485 |
| 1001 | sbtest | sbuser | 0x283AA9863F85EFC8 | SELECT DISTINCT c FROM sbtest4 WHERE id BETWEEN ? AND ? ORDER BY c | 3900 | 1542266823 | 1542267842 | 3363938 | 223 | 20208 |
| 1001 | sbtest | sbuser | 0xC2A4F66B0CA11A02 | SELECT c FROM sbtest4 WHERE id BETWEEN ? AND ? | 3900 | 1542266823 | 1542267842 | 2888626 | 144 | 220745 |
| 1001 | sbtest | sbuser | 0xC19480748AE79B4B | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c | 3873 | 1542266823 | 1542267842 | 3387251 | 223 | 17780 |
| 1001 | sbtest | sbuser | 0x9AF59B998A3688ED | SELECT c FROM sbtest2 WHERE id=? | 39230 | 1542266823 | 1542267842 | 23880648 | 58 | 73051 |
| 1001 | sbtest | sbuser | 0xBF001A0C13781C1D | SELECT c FROM sbtest1 WHERE id=? | 38730 | 1542266823 | 1542267842 | 23064060 | 66 | 54968 |
| 1001 | sbtest | sbuser | 0x16ADA60275E5EFAA | UPDATE sbtest2 SET k=k+? WHERE id=? | 3923 | 1542266823 | 1542267842 | 4077159 | 129 | 210837 |
| 1001 | sbtest | sbuser | 0x9D058B6F3BC2F754 | SELECT c FROM sbtest4 WHERE id=? | 39000 | 1542266823 | 1542267842 | 23666576 | 62 | 247702 |
| 1001 | sbtest | sbuser | 0xFAD1519E4760CBDE | BEGIN | 19399 | 1542266823 | 1542267842 | 23660721 | 42 | 24688 |
| 1001 | sbtest | sbuser | 0x713A0DB06FCE81E2 | DELETE FROM sbtest2 WHERE id=? | 3921 | 1542266823 | 1542267842 | 3411682 | 98 | 42979 |
| 1001 | sbtest | sbuser | 0x61511F703CBA5EA9 | UPDATE sbtest3 SET c=? WHERE id=? | 3826 | 1542266823 | 1542267842 | 3353539 | 108 | 87922 |
| 1001 | sbtest | sbuser | 0x0D3830CC26B680E5 | SELECT c FROM sbtest4 WHERE id BETWEEN ? AND ? ORDER BY c | 3900 | 1542266823 | 1542267842 | 3063405 | 172 | 17112 |
| 1001 | sbtest | sbuser | 0xC2E5FCE49337737D | UPDATE sbtest2 SET c=? WHERE id=? | 3922 | 1542266823 | 1542267842 | 3613263 | 106 | 37936 |
| 1001 | sbtest | sbuser | 0x0FEDA5F1D95F2FBA | SELECT c FROM sbtest3 WHERE id BETWEEN ? AND ? ORDER BY c | 3829 | 1542266823 | 1542267842 | 2942348 | 173 | 22736 |
| 1001 | sbtest | sbuser | 0x44BCB144058686EB | SELECT DISTINCT c FROM sbtest3 WHERE id BETWEEN ? AND ? ORDER BY c | 3829 | 1542266823 | 1542267842 | 3314628 | 223 | 20971 |
| 1001 | sbtest | sbuser | 0xE5D5BDF004370D6D | UPDATE sbtest3 SET k=k+? WHERE id=? | 3829 | 1542266823 | 1542267842 | 3835408 | 125 | 246168 |
| 1001 | sbtest | sbuser | 0x381AAD21F4326865 | SELECT c FROM sbtest2 WHERE id BETWEEN ? AND ? | 3923 | 1542266823 | 1542267842 | 2596934 | 144 | 14602 |
| 1001 | sbtest | sbuser | 0x4438008ADF85B3AE | SELECT SUM(K) FROM sbtest4 WHERE id BETWEEN ? AND ? | 3900 | 1542266823 | 1542267842 | 2768646 | 136 | 15846 |
| 1001 | sbtest | sbuser | 0x695FBF255DBEB0DD | COMMIT | 19344 | 1542266823 | 1542267842 | 236717934 | 1409 | 677436 |
| 1001 | sbtest | sbuser | 0x03744DC190BC72C7 | SELECT c FROM sbtest5 WHERE id=? | 38740 | 1542266823 | 1542267842 | 22732232 | 66 | 31644 |
| 1001 | sbtest | sbuser | 0x3138A29396F2C746 | SELECT SUM(K) FROM sbtest3 WHERE id BETWEEN ? AND ? | 3829 | 1542266823 | 1542267842 | 2702272 | 137 | 24684 |
| 1001 | sbtest | sbuser | 0x3929CFAB1E6B10B2 | UPDATE sbtest5 SET k=k+? WHERE id=? | 3874 | 1542266823 | 1542267842 | 3690598 | 128 | 73178 |
| 1001 | sbtest | sbuser | 0x290B92FD743826DA | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ? | 3873 | 1542266823 | 1542267842 | 2585710 | 137 | 15628 |
| 1001 | sbtest | sbuser | 0xB3A675D2CFC75492 | UPDATE sbtest5 SET c=? WHERE id=? | 3873 | 1542266823 | 1542267842 | 3169151 | 103 | 51781 |
| 1001 | sbtest | sbuser | 0x8B9F0559BA064E1C | SELECT SUM(K) FROM sbtest1 WHERE id BETWEEN ? AND ? | 3873 | 1542266823 | 1542267842 | 2674794 | 137 | 18206 |
| 1001 | sbtest | sbuser | 0xE0E39D08EEF48154 | DELETE FROM sbtest5 WHERE id=? | 3869 | 1542266823 | 1542267842 | 3065788 | 102 | 102785 |
| 1001 | sbtest | sbuser | 0x1288C33EA439294B | INSERT INTO sbtest5 (id, k, c, pad) VALUES (?, ?, ?, ?) | 3866 | 1542266823 | 1542267842 | 2553567 | 97 | 115583 |
| 1001 | sbtest | sbuser | 0xFB239BC95A23CA36 | UPDATE sbtest1 SET c=? WHERE id=? | 3871 | 1542266823 | 1542267842 | 3462745 | 102 | 131467 |
+-----------+------------+----------+--------------------+--------------------------------------------------------------------+------------+------------+------------+-----------+----------+----------+
47 rows in set (0.00 sec)
select hostgroup hg,sum_time,count_star,substr(digest_text,1,20) from stats_mysql_query_digest order by digest_text;
select hostgroup hg,sum_time,count_star,digest_text from stats_mysql_query_digest order by digest_text;
SELECT * FROM monitor.mysql_server_connect_log;
SELECT * FROM monitor.mysql_server_ping_log limit 10;
delete from monitor.mysql_server_connect_log;
delete from monitor.mysql_server_ping_log;
--下面這個sql在proxysql是專門清空stats_mysql_query_digest表的。
SELECT 1 FROM stats_mysql_query_digest_reset LIMIT 1;
select hostgroup hg,sum_time,count_star,digest_text from stats_mysql_query_digest order by digest_text;
--可用於測試MySQL的效能，但用於驗證讀寫分離不行
sysbench --test=/usr/share/sysbench/tests/include/oltp_legacy/oltp.lua --oltp-table-size=4000 --oltp-read-only=on --init-rng=on --num-threads=5 --max-requests=0 --oltp-dist-type=uniform --max-time=36 --mysql-user=sbuser --mysql-password='sbpass' --db-driver=mysql --mysql-port=6033 --mysql-table-engine=innodb --mysql-host=192.168.0.201 --mysql-db=sbtest --oltp-tables-count=5 --report-interval=1 run
--可用於驗證讀寫分離
sysbench --test=oltp_read_write --table-size=4000 --num-threads=5 --max-requests=0 --max-time=5 --mysql-user=sbuser --mysql-password='sbpass' --db-driver=mysql --mysql-port=6033 --mysql-host=192.168.0.201 --mysql-db=sbtest --report-interval=1 run