Gearman實現Mysql到Redis的資料同步

Gearman 是一個支援分散式的任務分發框架:
Gearman Job Server: Gearman 核心程式,需要編譯安裝並以守護程序形式執行在後臺。
Gearman Client:可以理解為任務的請求者。
Gearman Worker:任務的真正執行者,一般需要自己編寫具體邏輯並通過守護程序方式
執行,Gearman Worker 接收到 Gearman Client 傳遞的任務內容後,會按順序處理。

大致流程:

1)下面要編寫的 mysql 觸發器,就相當於 Gearman 的客戶端。

2)修改表,插入表就相當於直接下發任務。

3)再通過 lib_mysqludf_json UDF 庫函式將關係資料對映為 JSON 格式

4)再通過 gearman-mysql-udf 外掛將任務加入到 Gearman 的任務佇列中

5)最後通過redis_worker.php,也就是 Gearman 的 worker 端來完成 redis 資料庫的更新。

配置過程：

1、安裝gearmand並開啟服務

[[email protected] ~]# cd redis/
[[email protected] redis]# yum install -y gearmand-1.1.8-2.el6.x86_64.rpm libgearman-* libevent-*
[
 
[email protected] redis]# /etc/init.d/gearmand start 

示圖：安裝過程存在依賴性，同步安裝

[[email protected] redis]# vim worker.php    ##Master的ip
[[email protected] redis]# cp worker.php /usr/local/

2、編譯、安裝gearman

[[email protected] redis]# tar zxf gearman-1.1.2.tgz
[[email protected] redis]# cd gearman-1.1.2 
[
 
[email protected] gearman-1.1.2]# phpize 
[[email protected] gearman-1.1.2]# ./configure 
[[email protected] gearman-1.1.2]# make && make install 
[[email protected] gearman-1.1.2]# cd /etc/php.d
[[email protected] php.d]# cp redis.ini gearman.ini
[[email protected] php.d]# vim gearman.ini 
extension=gearman.so
[[email protected] php.d]# /etc/init.d/php-fpm reload 
[[email protected] php.d]# php -m     
gearman 
mysql
redis

3、生成lib_mysqludf_json.so模組

[[email protected] redis]# scp lib_mysqludf_json-master.zip gearman-mysql-udf-0.6.tar.gz server3:
[[email protected] redis]# scp libevent-* server3:
[[email protected] redis]# scp libgearman-* server3:
[[email protected] ~]# yum install -y mysql-devel
[[email protected] ~]# yum install -y gcc
[[email protected] ~]# unzip lib_mysqludf_json-master.zip 
[[email protected] ~]# cd lib_mysqludf_json-master
[[email protected] lib_mysqludf_json-master]# ls
lib_mysqludf_json.c     lib_mysqludf_json.so   README.md
lib_mysqludf_json.html  lib_mysqludf_json.sql
[[email protected] lib_mysqludf_json-master]# gcc $(mysql_config --cflags) -shared -fPIC -o lib_mysqludf_json.so lib_mysqludf_json.c 
[[email protected] lib_mysqludf_json-master]# cp lib_mysqludf_json.so /usr/lib64/mysql/plugin/

4、註冊 json 函式

[[email protected] ~]# /etc/init.d/mysqld start
[[email protected] ~]# mysql
mysql> show global variables like 'plugin_dir';
+---------------+-------------------------+
| Variable_name | Value                   |
+---------------+-------------------------+
| plugin_dir    | /usr/lib64/mysql/plugin |
+---------------+-------------------------+
mysql> create function json_object returns string soname 'lib_mysqludf_json.so';
mysql> select * from mysql.func;
+-------------+-----+----------------------+----------+
| name        | ret | dl                   | type     |
+-------------+-----+----------------------+----------+
| json_object |   0 | lib_mysqludf_json.so | function |
+-------------+-----+----------------------+----------+

5、註冊UDF函式

[[email protected] ~]# yum install -y libevent-*
[[email protected] ~]# yum install -y libgearman-* 
[[email protected] ~]# tar zxf gearman-mysql-udf-0.6.tar.gz 
[[email protected] ~]# cd gearman-mysql-udf-0.6
[[email protected] gearman-mysql-udf-0.6]# ./configure --libdir=/usr/lib64/mysql/plugin/
[[email protected] gearman-mysql-udf-0.6]# make && make install 

6、指定 gearman 的服務資訊

[[email protected] ~]# mysql
mysql> create function gman_do_background returns string soname 'libgearman_mysql_udf.so';
mysql> create function gman_servers_set returns string soname 'libgearman_mysql_udf.so';
mysql> select * from mysql.func;
+--------------------+-----+-------------------------+----------+
| name               | ret | dl                      | type     |
+--------------------+-----+-------------------------+----------+
| json_object        |   0 | lib_mysqludf_json.so    | function |
| gman_do_background |   0 | libgearman_mysql_udf.so | function |
| gman_servers_set   |   0 | libgearman_mysql_udf.so | function |
+--------------------+-----+-------------------------+----------+
mysql> select gman_servers_set('172.25.51.1:4730');
+--------------------------------------+
| gman_servers_set('172.25.51.1:4730') |
+--------------------------------------+
| 172.25.51.1:4730                     |
+--------------------------------------+

7、編寫 mysql 觸發器

[[email protected] ~]# vim test.sql 
[[email protected] ~]# mysql < test.sql 
mysql> show triggers from test;
[[email protected] ~]# mysql
mysql> show triggers from test;
+-------------+--------+-------+----------------------------------------------------------------------------------------------------------------+--------+---------+----------+----------------+----------------------+----------------------+--------------------+
| Trigger     | Event  | Table | Statement                                                                                                      | Timing | Created | sql_mode | Definer        | character_set_client | collation_connection | Database Collation |
+-------------+--------+-------+----------------------------------------------------------------------------------------------------------------+--------+---------+----------+----------------+----------------------+----------------------+--------------------+
| datatoredis | UPDATE | test  | BEGIN
    SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`)); 
  END | AFTER  | NULL    |          | [email protected] | latin1               | latin1_swedish_ci    | latin1_swedish_ci  |
+-------------+--------+-------+----------------------------------------------------------------------------------------------------------------+--------+---------+----------+----------------+----------------------+----------------------+--------------------+

示圖：test資料庫內容

8、測試：

server1：開啟nginx、php-fpm

server3：更改test資料庫資訊

[[email protected] ~]# mysql
mysql> use test
mysql> update test set name='hhh' where id=1;

客戶端：