1. 程式人生 > >PostgreSQL 同步流複製延遲測試(二)

PostgreSQL 同步流複製延遲測試(二)

1主2從SR同步流複製測

  • 搭建環境

伺服器 | Role
|- | :-: | -: |
10.10.56.16 | master
10.10.56.17 | slave1
10.10.56.19 | slave2

  • 16查詢狀態
pocdb=# SELECT client_addr,application_name,sync_state FROM pg_stat_replication;
 client_addr | application_name | sync_state
-------------+------------------+------------
 10.10
.56.17 | slave1 | sync 10.10.56.19 | slave2 | potential (2 rows) pocdb=#

當從庫大於兩臺機器時同步狀態為 potential ,表示可能會提升為 master

  • 查詢使用者
pocdb=# \du+
                                          List of roles
 Role name |                         Attributes                         | Member of | Description
-----------+------------------------------------------------------------+-----------+-------------
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {} | repl | Replication | {} | pocdb=#
  • 16建立同步測試表 synctest 和序列
pocdb=# create table synctest (id bigint primary key ,number bigint,date timestamp default
now());
CREATE TABLE pocdb=# create sequence seq_synctest increment by 1 minvalue 1 maxvalue 99999999999999 cache 50 no cycle; CREATE SEQUENCE pocdb=#
  • 查看錶大小
pocdb=# \d+
                           List of relations
 Schema |     Name     |   Type   |  Owner   |    Size    | Description
--------+--------------+----------+----------+------------+-------------
 public | seq_synctest | sequence | postgres | 8192 bytes |
 public | synctest     | table    | postgres | 0 bytes    |
(2 rows)

pocdb=#

此時17、19伺服器自動同步該表,可分別去查詢

  • 16伺服器編寫插入指令碼
postgres@clw-db1:/pgdata/10/poc/scripts> vi bench_script_for_insert_20180717.sql
[email protected]:/pgdata/10/poc/scripts> cat bench_script_for_insert_20180717.sql
\set number random(1, 100000000000000000000000)
INSERT INTO synctest(id,number) VALUES (nextval('seq_synctest'),:number);
postgres@clw-db1:/pgdata/10/poc/scripts>
  • 16、17、19 編寫監控延遲指令碼
[email protected]:/pgdata/10/poc/scripts> cat monior_syncSR_relay.sh
#!/bin/bash

/opt/pgsql-10/bin/psql pocdb<<EOF
select now();
select client_addr, application_name, write_lag, flush_lag, replay_lag \
from pg_stat_replication where usename='l_repl' and application_name='slave1';
 \q
EOF
[email protected]:/pgdata/10/poc/scripts>
  • 啟動pgbench進行壓測
[email protected]-db1:/pgdata/10/poc/scripts> /opt/pgsql-10/bin/pgbench -T 1200 -j 600 -c 500  -f  bench_script_for_insert_20180717.sql pocdb
  • 編寫slave1 的資料查詢指令碼
for i in {1..1000000000}
do
/pgdata/10/poc/scripts/monior_syncSR_relay_slave1.sh >> syncSR_relay_slave1_result
sleep 10
done
for i in {1..1000000000}
do
/pgdata/10/poc/scripts/monior_syncSR_relay_slave2.sh >> syncSR_relay_slave2_result
sleep 10
done
[email protected]:/pgdata/10/poc/scripts> cat query_count_slave1.sh
#!/bin/bash

/opt/pgsql-10/bin/psql pocdb<<EOF
select now();
select max(id) from synctest;
 \q
EOF
[email protected]:/pgdata/10/poc/scripts>
for i in {1..1000000000}
do
/pgdata/10/poc/scripts/query_count_result.sh >> query_count_sum
sleep 10
done

查詢插入數量指令碼

[email protected]:/pgdata/10/poc/scripts> cat query_count_result.sh
#!/bin/bash

/opt/pgsql-10/bin/psql pocdb<<EOF
select now();
select max(id) from synctest;
 \q
EOF
[email protected]:/pgdata/10/poc/scripts>
  • 編寫記憶體、網路、IO效能監控指令碼
/home/super/pgsoft/nmon_x86_64_sles11 -f  -c 150  -s 10 
  • 17 查詢數量指令碼
for i in {1..1000000000}
do
/pgdata/10/poc/scripts/query_count_slave1.sh >> query_count_slave1_sum
sleep 10
done
  • 17 效能監控指令碼
/home/pgsoft/nmon_x86_64_sles11 -f  -c 150  -s 10 
  • 19 查詢數量指令碼
for i in {1..1000000000}
do
/pgdata/10/poc/scripts/query_count_slave2.sh >> query_count_slave2_sum
sleep 10
done
  • 19效能監控
/home/pgsoft/nmon_x86_64_sles11 -f  -c 150  -s 10 

-16測試結果

could not connect to server: Resource temporarily unavailable
        Is the server running locally and accepting
        connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
transaction type: bench_script_for_insert_20180717.sql
scaling factor: 1
query mode: simple
number of clients: 500
number of threads: 500
duration: 1200 s
number of transactions actually processed: 9723149
latency average = 61.736 ms
tps = 8098.953578 (including connections establishing)
tps = 8099.988751 (excluding connections establishing)
[email protected]:/pgdata/10/poc/scripts>

pocdb=# \d+
                           List of relations
 Schema |     Name     |   Type   |  Owner   |    Size    | Description
--------+--------------+----------+----------+------------+-------------
 public | seq_synctest | sequence | postgres | 8192 bytes |
 public | synctest     | table    | postgres | 377 MB     |
(2 rows)

pocdb=#

1主2從同步測試結果

測試背景:伺服器10.10.56.16,10.10.56.17,10.10.56.19 CPU 8核 記憶體200G 網路傳輸3M/s

測試方法: 20 分鐘 600執行緒 500客戶端連線數 不停往 16master 伺服器寫入資料

測試結果:CPU 佔用率 80% ,磁碟 I/O 23MB/s ,TPS:7419 ,資料量為:1880萬,
同步時延大約為 3毫秒左右

  • 壓測指令碼(-T 1200秒 -j 600 執行緒 -c 客戶端連併發接數500 -f 輸出壓測結果)
/opt/pgsql-10/bin/pgbench -T 1200 -j 600 -c 500  -f  bench_script_for_insert_20180717.sql pocdb
  • 第一次測試(1主2從同步SR)
transaction type: bench_script_for_insert_20180717.sql
scaling factor: 1
query mode: simple
number of clients: 500
number of threads: 500
duration: 1200 s
number of transactions actually processed: 8906813
latency average = 67.391 ms
tps = 7419.410395 (including connections establishing)
tps = 7420.536021 (excluding connections establishing)
[email protected]:/pgdata/10/poc/scripts>
  • 第二次測試(1主2從同步SR)
[email protected]-db1:/pgdata/10/poc/scripts> /opt/pgsql-10/bin/pgbench -T 1200 -j 600 -c 500  -f bench_script_for_insert_20180717.sql pocdb
query mode: simple
number of clients: 500
number of threads: 500
duration: 1200 s
number of transactions actually processed: 8239738
latency average = 72.848 ms
tps = 6863.613524 (including connections establishing)
tps = 6865.035476 (excluding connections establishing)
[email protected]:/pgdata/10/poc/scripts>
  • 查詢主從資料延遲,主要觀察引數(write_lag、flush_lsn、replay_lag)
pocdb=# select * from pg_stat_replication;
-[ RECORD 1 ]----+------------------------------
pid              | 23564
usesysid         | 16393
usename          | repl
application_name | slave2
client_addr      | 10.10.56.19
client_hostname  |
client_port      | 52820
backend_start    | 2018-05-16 17:43:23.726216+08
backend_xmin     |
state            | streaming
sent_lsn         | 1/7D4FF030
write_lsn        | 1/7D4FF030
flush_lsn        | 1/7D4FEF78
replay_lsn       | 1/7D4FEF78
write_lag        | 00:00:00.003057
flush_lag        | 00:00:00.003057
replay_lag       | 00:00:00.003057
sync_priority    | 2
sync_state       | potential
-[ RECORD 2 ]----+------------------------------
pid              | 23562
usesysid         | 16393
usename          | repl
application_name | slave1
client_addr      | 10.10.56.17
client_hostname  |
client_port      | 33647
backend_start    | 2018-05-16 17:43:17.371715+08
backend_xmin     |
state            | streaming
sent_lsn         | 1/7D524110
write_lsn        | 1/7D523F30
flush_lsn        | 1/7D523570
replay_lsn       | 1/7D523570
write_lag        | 00:00:00.000329
flush_lag        | 00:00:00.000329
replay_lag       | 00:00:00.000329
sync_priority    | 1
sync_state       | sync

pocdb=# select * from pg_stat_replication;
-[ RECORD 1 ]----+------------------------------
pid              | 23564
usesysid         | 16393
usename          | repl
application_name | slave2
client_addr      | 10.10.56.19
client_hostname  |
client_port      | 52820
backend_start    | 2018-05-16 17:43:23.726216+08
backend_xmin     |
state            | streaming
sent_lsn         | 1/7EB37850
write_lsn        | 1/7EB36F00
flush_lsn        | 1/7EB34EE0
replay_lsn       | 1/7EB34EE0
write_lag        | 00:00:00.000841
flush_lag        | 00:00:00.000841
replay_lag       | 00:00:00.000841
sync_priority    | 2
sync_state       | potential
-[ RECORD 2 ]----+------------------------------
pid              | 23562
usesysid         | 16393
usename          | repl
application_name | slave1
client_addr      | 10.10.56.17
client_hostname  |
client_port      | 33647
backend_start    | 2018-05-16 17:43:17.371715+08
backend_xmin     |
state            | streaming
sent_lsn         | 1/7EB37F80
write_lsn        | 1/7EB27B08
flush_lsn        | 1/7EB27B08
replay_lsn       | 1/7EB27B08
write_lag        | 00:00:00.001525
flush_lag        | 00:00:00.003983
replay_lag       | 00:00:00.012568
sync_priority    | 1
sync_state       | sync
  • 查詢資料量
pocdb=# \d+
                           List of relations
 Schema |     Name     |   Type   |  Owner   |    Size    | Description
--------+--------------+----------+----------+------------+-------------
 public | seq_synctest | sequence | postgres | 8192 bytes |
 public | synctest     | table    | postgres | 377 MB     |
(2 rows)

pocdb=# select max(id) from synctest;
   max
----------
 18812501
(1 row)

pocdb=#

單庫測試

  • 48伺服器建立資料庫 pocdb 和表 synctest
postgres=# create database pocdb;
CREATE DATABASE

pocdb=# create table synctest (id bigint primary key ,number bigint,date timestamp default now());
CREATE TABLE
pocdb=# create sequence seq_synctest increment by 1 minvalue 1 maxvalue 99999999999999 cache 50 no cycle;
CREATE SEQUENCE
pocdb=# \d+
                           List of relations
 Schema |     Name     |   Type   |  Owner   |    Size    | Description
--------+--------------+----------+----------+------------+-------------
 public | seq_synctest | sequence | postgres | 8192 bytes |
 public | synctest     | table    | postgres | 0 bytes    |
(2 rows)

pocdb=#
  • 48效能監控
/home/postgres/pgsoft/nmon_x86_64_sles11 -c 180 -f -s 10
  • 48 查詢數量指令碼
for i in {1..100000000}
do
/pgdata/10/scripts/query_count_result.sh >> query_count_slave2_sum
sleep 10
done
  • 48 壓測指令碼
 /opt/pgsql-10/bin/pgbench -T 1200 -j 800 -c 400  -f bench_script_for_insert_20180717.sql pocdb

單例項庫測試

  • 測試背景: 伺服器 : 10.10.56.17 cpu:8核 記憶體:128GB

  • 測試過程: 在伺服器上執行 20分鐘寫入資料庫測試,500 執行緒數 500 客戶端連線數

    • 測試結果: 寫入資料庫共3500萬 ,平均響應時間 17.125ms ,TPS 為29000 ,CPU佔用95% 磁碟寫速率:35MB/s ,網路傳輸速率:3MB/s
  • 17 伺服器 單庫壓測結果

    transaction type: bench_script_for_insert_20180717.sql
    scaling factor: 1
    query mode: simple
    number of clients: 500
    number of threads: 500
    duration: 1200 s
    number of transactions actually processed: 35047438
    latency average = 17.125 ms
    tps = 29197.286164 (including connections establishing)
    tps = 29205.049038 (excluding connections establishing)
    [email protected]:/pgdata/estest10/scripts>
  • 查詢資料
    “`
    pocdb1=# \d+
    List of relations
    Schema | Name | Type | Owner | Size | Description
    ——–+————–+———-+———-+————+————-
    public | seq_synctest | sequence | postgres | 8192 bytes |
    public | synctest | table | postgres | 1747 MB |
    (2 rows)

pocdb1=# select max(id) from synctest;

max

35947438
(1 row)

pocdb1=#
“`

主從與單庫效能對比

結果分析:單褲寫入效能更高,大約為1主2從的2倍,TPS 單庫大約為主從的4倍,單庫磁碟寫入速率更高,以上僅為測試資料,僅供參考。

從節點宕機測試

  • 測試過程: kill 掉從節點,模擬意外宕機,往主節點寫入資料

  • 測試結果: 同步流複製時,從節點宕機會影響主庫,主庫進行寫入和更新時會被夯住。

  • kill掉17 slave1,主庫插入資料,會夯住。

pocdb=# insert into synctest(age,date)values(nextval(seq_synctest'),28701752,now());
pocdb'#

一主一從

  • 測試背景:伺服器 : 10.10.56.17 cpu:8核 記憶體:128GB
  • 測試過程: 在伺服器上執行 20分鐘寫入資料庫測試,500 執行緒數 500 客戶端連線數
  • 測試結果:寫入資料:3890萬 CPU佔用:80% 磁碟寫:25Mb/s 延遲:2毫秒 ,TPS :8288

    測試結果

transaction type: bench_script_for_insert_20180717.sql
scaling factor: 1
query mode: simple
number of clients: 400
number of threads: 400
duration: 1200 s
number of transactions actually processed: 9946727
latency average = 48.258 ms
tps = 8288.726962 (including connections establishing)
tps = 8289.701749 (excluding connections establishing)
[email protected]:/pgdata/10/poc/scripts>
  • 查詢插入資料量
pocdb=# \d+
                           List of relations
 Schema |     Name     |   Type   |  Owner   |    Size    | Description
--------+--------------+----------+----------+------------+-------------
 public | seq_synctest | sequence | postgres | 8192 bytes |
 public | synctest     | table    | postgres | 496 MB     |
(2 rows)

pocdb=# select max(id) from synctest;
   max
----------
 38918501
(1 row)

pocdb=#
  • 查詢延遲,觀察引數(write_lag,flush_lag,replay_lag)
pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.002566 | 00:00:00.002566 | 00:00:00.002566 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.000774 | 00:00:00.000774 | 00:00:00.000774 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.000648 | 00:00:00.000648 | 00:00:00.000648 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.000408 | 00:00:00.000408 | 00:00:00.000408 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.002627 | 00:00:00.002666 | 00:00:00.008609 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.000593 | 00:00:00.000593 | 00:00:00.000593 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.003211 | 00:00:00.009313 | 00:00:00.015688 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.001078 | 00:00:00.001078 | 00:00:00.001078 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.021852 | 00:00:00.034961 | 00:00:00.035026 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag   | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.000829 | 00:00:00.002209 | 00:00:00.00334 | sync       | streaming
(1 row)

pocdb=# select client_addr,usename, application_name, write_lag, flush_lag, replay_lag,sync_state,state from pg_stat_replication;
 client_addr | usename | application_name |    write_lag    |    flush_lag    |   replay_lag    | sync_state |   state
-------------+---------+------------------+-----------------+-----------------+-----------------+------------+-----------
 10.10.56.19 | repl    | slave2           | 00:00:00.054129 | 00:00:00.054134 | 00:00:00.054964 | sync       | streaming
(1 row)

效能對比

測試對比:1主1從寫 入效能和 單庫寫 效能相差不大,20分鐘寫入資料量大約都為3700萬,1主兩從寫入效能較差,資料只有它倆的 一半,延遲比 1主1從 高點。以上僅為測試資料,僅供參考,沒有達到極限的情況

相關推薦

no