zabbix3.2分區表配置

阿新 • • 發佈：2018-03-21

zabbix3.2分區表配置 mysql分區表配置

添加了將近1000臺主機，數據庫中的歷史數據越來越多了。出現了zabbix自帶的housekeeper清理歷史數據的時候，造成查數據慢並且觸發了很多報警信息（比如Zabbix housekeeper processes more than 75% busy）。

housekeeper默認一小時清理一次舊的數據。在server配置文件中這兩行可以定義：
HousekeepingFrequency=1 zabbix執行Housekeeping的頻率，單位為hours
MaxHousekeeperDelete=500 每次最多刪除歷史數據的行

但是housekeeper清理過中，會導致數據庫負載增高，從而影響讀寫性能。

所以我們對幾個歷史數據表做分區表。按照時間（每天）為單位，把歷史數據存到各個分區表中，這樣做能加快查詢速度、快速清理過去一時間的歷史數據（刪除分區表）

1、先查看每個表所占容量和行數，可以看到history的表很大

MariaDB [zabbix]> select table_name, (data_length+index_length)/1024/1024 as total_mb, table_rows from information_schema.tables where table_schema='zabbix';
+----------------------------+---------------+------------+
| table_name                 | total_mb      | table_rows |
+----------------------------+---------------+------------+
| acknowledges               |    0.06250000 |          0 |
| actions                    |    0.04687500 |         14 |
| alerts                     |    5.60937500 |       7316 |
| application_discovery      |    0.04687500 |          0 |
| application_prototype      |    0.04687500 |          1 |
| application_template       |    2.37500000 |      13963 |
| applications               |    1.96875000 |      14035 |
| auditlog                   |   18.54687500 |     100270 |
| auditlog_details           |    0.42187500 |       2321 |
| autoreg_host               |    0.03125000 |          0 |
| conditions                 |    0.03125000 |         61 |
| config                     |    0.04687500 |          1 |
| corr_condition             |    0.03125000 |          0 |
| corr_condition_group       |    0.03125000 |          0 |
| corr_condition_tag         |    0.01562500 |          0 |
| corr_condition_tagpair     |    0.01562500 |          0 |
| corr_condition_tagvalue    |    0.01562500 |          0 |
| corr_operation             |    0.03125000 |          0 |
| correlation                |    0.04687500 |          0 |
| dbversion                  |    0.01562500 |          1 |
| dchecks                    |    0.03125000 |          1 |
| dhosts                     |    0.03125000 |          0 |
| drules                     |    0.04687500 |          1 |
| dservices                  |    0.04687500 |          0 |
| escalations                |    0.03125000 |        169 |
| event_recovery             |   14.10937500 |      16990 |
| event_tag                  |    0.03125000 |          0 |
| events                     |   41.09375000 |      35770 |
| expressions                |    0.03125000 |          6 |
| functions                  |   10.78125000 |      56230 |
| globalmacro                |    0.03125000 |          1 |
| globalvars                 |    0.01562500 |          0 |
| graph_discovery            |    0.03125000 |         88 |
| graph_theme                |    0.03125000 |          2 |
| graphs                     |    6.09375000 |      17348 |
| graphs_items               |   12.20312500 |      73151 |
| group_discovery            |    0.03125000 |          0 |
| group_prototype            |    0.06250000 |          7 |
| groups                     |    0.03125000 |         19 |
| history                    | 1873.62500000 |   17753848 |
| history_log                |    0.03125000 |          0 |
| history_str                |    2.07812500 |       3330 |
| history_text               |    0.07812500 |        594 |
| history_uint               | 5980.10937500 |   38661029 |
| host_discovery             |    0.04687500 |          2 |
| host_inventory             |    0.01562500 |          0 |
| hostmacro                  |    0.03125000 |          0 |
| hosts                      |    0.48437500 |        725 |
| hosts_groups               |    0.15625000 |        715 |
| hosts_templates            |    0.15625000 |        859 |
| housekeeper                |    0.01562500 |        204 |
| httpstep                   |    0.03125000 |          3 |
| httpstepitem               |    0.04687500 |          9 |
| httptest                   |    0.07812500 |          3 |
| httptestitem               |    0.04687500 |          9 |
| icon_map                   |    0.04687500 |          0 |
| icon_mapping               |    0.04687500 |          0 |
| ids                        |    0.01562500 |         43 |
| images                     |    1.53125000 |        191 |
| interface                  |    0.18750000 |        671 |
| interface_discovery        |    0.03125000 |          0 |
| item_application_prototype |    0.04687500 |          1 |
| item_condition             |    0.21875000 |       1291 |
| item_discovery             |    0.75000000 |       5656 |
| items                      |   47.15625000 |      98946 |
| items_applications         |   17.21875000 |     119577 |
| maintenances               |    0.04687500 |          0 |
| maintenances_groups        |    0.04687500 |          0 |
| maintenances_hosts         |    0.04687500 |          0 |
| maintenances_windows       |    0.04687500 |          0 |
| mappings                   |    0.03125000 |        143 |
| media                      |    0.04687500 |         15 |
| media_type                 |    0.03125000 |          6 |
| opcommand                  |    0.03125000 |          4 |
| opcommand_grp              |    0.04687500 |          0 |
| opcommand_hst              |    0.04687500 |          4 |
| opconditions               |    0.03125000 |          0 |
| operations                 |    0.03125000 |         29 |
| opgroup                    |    0.04687500 |          1 |
| opinventory                |    0.01562500 |          0 |
| opmessage                  |    0.03125000 |         23 |
| opmessage_grp              |    0.04687500 |          5 |
| opmessage_usr              |    0.04687500 |         15 |
| optemplate                 |    0.04687500 |          1 |
| problem                    |    4.32812500 |      17405 |
| problem_tag                |    0.04687500 |          0 |
| profiles                   |    0.34375000 |       1641 |
| proxy_autoreg_host         |    0.03125000 |          0 |
| proxy_dhistory             |    0.03125000 |          0 |
| proxy_history              |    0.03125000 |          0 |
| regexps                    |    0.03125000 |          5 |
| rights                     |    0.04687500 |         19 |
| screen_user                |    0.04687500 |          0 |
| screen_usrgrp              |    0.04687500 |          0 |
| screens                    |    0.04687500 |         14 |
| screens_items              |    0.03125000 |         51 |
| scripts                    |    0.06250000 |          3 |
| service_alarms             |    0.04687500 |          0 |
| services                   |    0.03125000 |          0 |
| services_links             |    0.04687500 |          0 |
| services_times             |    0.03125000 |          0 |
| sessions                   |    2.01562500 |       5953 |
| slides                     |    0.04687500 |          1 |
| slideshow_user             |    0.04687500 |          0 |
| slideshow_usrgrp           |    0.04687500 |          0 |
| slideshows                 |    0.04687500 |          1 |
| sysmap_element_url         |    0.03125000 |          0 |
| sysmap_url                 |    0.03125000 |          0 |
| sysmap_user                |    0.04687500 |          0 |
| sysmap_usrgrp              |    0.04687500 |          0 |
| sysmaps                    |    0.07812500 |          1 |
| sysmaps_elements           |    0.09375000 |          1 |
| sysmaps_link_triggers      |    0.04687500 |          0 |
| sysmaps_links              |    0.06250000 |          0 |
| task                       |    0.01562500 |          0 |
| task_close_problem         |    0.01562500 |          0 |
| timeperiods                |    0.01562500 |          0 |
| trends                     |  133.12500000 |     590089 |
| trends_uint                |  141.07812500 |     472377 |
| trigger_depends            |    0.04687500 |         65 |
| trigger_discovery          |    0.03125000 |        106 |
| trigger_tag                |    0.03125000 |          0 |
| triggers                   |   18.31250000 |      46691 |
| users                      |    0.03125000 |         10 |
| users_groups               |    0.04687500 |         11 |
| usrgrp                     |    0.03125000 |         12 |
| valuemaps                  |    0.03125000 |         17 |
+----------------------------+---------------+------------+
127 rows in set (1.20 sec)

2、下載分區腳本（適合zabbix 3.2，謝謝網友寫的）

[root@localhost ~]# wget https://dl.cactifans.com/zabbix/partitiontables_gt_zbx34.sh

腳本默認詳情數據保留30天，趨勢數據保留12個月，如需修改，請修改以下內容：
daily_history_min=30
monthly_history_min=12

腳本默認連接數據庫信息，更改成你的：
DBHOST=localhost
DBUSER=zabbix
DBPASS=zabbix

賦予執行權限：

[root@localhost ~]# chmod +x partitiontables_gt_zbx34.sh

腳本內容如下：

#!/bin/bash
#
# This script will partition your zabbix database to improve the efficiency.
# It will also create stored procedures to do the necessary housekeeping,
# and create a cronjob to do this on a daily basis
#
# This script inspired by the following:
#    http://zabbixzone.com/zabbix/partitioning-tables/
#
# While the basic SQL is from the above page, this script both creates the necessary 
# SQL for the desired tables, and can create new partitions as the time goes on
# assuming that the cronjob has been properly entered.
#

#
# Who to email with cron output
#
EMAIL="root@localhost"

#
# How long to keep the daily history
#
daily_history_min=30

#
# How long to keep the monthly history (months)
#
monthly_history_min=12

#
# Years to create the monthly partitions for
#
first_year=`date +"%Y"`
last_year=$first_year
cur_month=`date +"%m"|sed 's/^0*//'`
if [ $cur_month -eq 12 ]; then
    last_year=$((first_year+1))
    cur_month=1
fi

y=`date +"%Y"`

SQL="/tmp/partition.sql"
PATHTOCRON="/usr/local/zabbix/cron.d"
PATHTOMAILBIN="/usr/bin/mail"
DUMP_FILE=/tmp/zabbix.sql

function usage {
cat <<_EOF_

$0    [-h host][-u user][-p password][-d min_days][-y startyear][-n][-s][-e email_address][-b]

    -h host            database host
    -u user            db user
    -p password        user password
    -d min_days        Minimum number of days of history to keep (default: $daily_history_min)
    -m min_months        Minimum number of months to keep trends (default: $monthly_history_min)
    -y startyear        First year to set up with partitions
    -n noninteractive    Run without questions - careful, make sure you know what is going to happen. Needs my.cnf with correct permissions.
    -b backup        Create backup of DB in $DUMP_FILE before alterations (only works with non-interactive mode, -n)
    -s simulate        Create SQL file that would be executed for examination ($SQL)
    -e email        Email address to receive partition update report (default: $EMAIL)


After running this script, don't forget to disable housekeeping if
you didn't have the script disable it, and add the following cronjob

    ### Option: DisableHousekeeping
    #       If set to 1, disables housekeeping.
    #
    # Mandatory: no
    # Range: 0-1
    ################### Uncomment and change the following line to 1 in 
    ################### Then restart the zabbix server
    DisableHousekeeping=1


Cron job

0 0 * * *  $PATHTOCRON/housekeeping.sh


_EOF_
    exit
}

DBHOST=localhost
DBUSER=zabbix
DBPASS=zabbix
SIMULATE=0
NONINTERACTIVE=0
BACKUP=0
while getopts "m:nsbe:h:u:p:d:y:?h" flag; do
    case $flag in
        h)    DBHOST=$OPTARG    ;;
        u)    DBUSER=$OPTARG    ;;
        p)    DBPASS=$OPTARG    ;;
        e)    EMAIL=$OPTARG     ;;
        s)    SIMULATE=1        ;;
        n)    NONINTERACTIVE=1  ;;
        b)    BACKUP=1          ;;
        d)    h=$OPTARG
            if [ $h -gt 0 ] 2>/dev/null; then
                daily_history_min=$h
            else
                echo "Invalid daily history min, exiting"
                exit 1
            fi
            ;;
        m)    h=$OPTARG
            if [ $h -gt 0 ] 2>/dev/null; then
                monthly_history_min=$h
            else
                echo "Invalid monthly history min, exiting"
                exit 1
            fi
            ;;

        y)    yy=$OPTARG
            if [ $yy -lt $y -a $yy -gt 2000 ] 2>/dev/null; then
                first_year=$yy
            else
                echo "Invalid year, exiting"
                exit 1
            fi
            ;;
        ?|h)    usage ;;
    esac
done
shift $((OPTIND-1))

if [ $NONINTERACTIVE != 1 ]; then
    echo "Ready to partition tables."
fi

if [ $SIMULATE = 0 ]; then
    if [ $NONINTERACTIVE = 1 ]; then
        mysql -B -h $DBHOST -e "GRANT CREATE ROUTINE ON zabbix.* TO '$DBUSER'@'localhost';"
#        echo "GRANT LOCK TABLES ON zabbix.* TO '${DBUSER}'@'${DBHOST}' IDENTIFIED BY '${DBPASS}';" | mysql -h${DBHOST} -u${DBADMINUSER} --password=${DBADMINPASS}
                mysql -h $DBHOST -e "GRANT LOCK TABLES ON zabbix.* TO '$DBUSER'@'$DBHOST' IDENTIFIED BY '$DBPASS';"
        if [ $BACKUP = 1 ]; then
            mysqldump --opt -h $DBHOST -u $DBUSER -p$DBPASS zabbix --result-file=$DUMP_FILE
            rc=$?
            if [ $rc -ne 0 ]; then
                echo "Error during mysqldump, exit code: $rc"
            fi
        fi
    else
        echo -e "\nReady to update permissions of Zabbix user to create routines\n"
        echo -n "Enter root DB user: "
        read DBADMINUSER
        echo -n "Enter $DBADMINUSER password: "
        read DBADMINPASS
        mysql -B -h $DBHOST -u $DBADMINUSER -p$DBADMINPASS -e "GRANT CREATE ROUTINE ON zabbix.* TO '$DBUSER'@'localhost';"
        echo -e "\n"

        echo -ne "\nDo you want to backup the database (recommended) (Y/n): "
        read yn
        if [ "$yn" != "n" -a "$yn" != "N" ]; then
            echo -e "\nEnter output file, press return for default of $DUMP_FILE"
            read df
            [ "$df" != "" ] && DUMP_FILE=$df

            #
            # Lock tables is needed for a good mysqldump
            #
            echo "GRANT LOCK TABLES ON zabbix.* TO '${DBUSER}'@'${DBHOST}' IDENTIFIED BY '${DBPASS}';" | mysql -h${DBHOST} -u${DBADMINUSER} --password=${DBADMINPASS}

            mysqldump --opt -h ${DBHOST} -u ${DBUSER} -p${DBPASS} zabbix --result-file=${DUMP_FILE}
            rc=$?
            if [ $rc -ne 0 ]; then
                echo "Error during mysqldump, rc: $rc"
                echo "Do you wish to continue (y/N): "
                read yn
                [ "yn" != "y" -a "$yn" != "Y" ] && exit
            else
                echo "Mysqldump succeeded!, proceeding with upgrade..."
            fi
        else
            echo "Are you certain you have a backup (y/N): "
            read yn
            [ "$yn" != 'y' -a "$yn" != "Y" ] && exit
        fi
    fi
fi

if [ $NONINTERACTIVE = 1 ]; then
    yn='y'
else
    echo -e "\n\nReady to proceed:"

    echo -e "\nStarting yearly partioning at: $first_year"
    echo "and ending at: $last_year"
    echo "With $daily_history_min days of daily history"
    echo -e "\n\nReady to proceed (Y/n): "
    read yn
    [ "$yn" = 'n' -o "$yn" = "N" ] && exit
fi


DAILY="history history_log history_str history_text history_uint"
DAILY_IDS="itemid id itemid id itemid"

MONTHLY="trends trends_uint" 
#"acknowledges alerts auditlog events service_alarms"
MONTHLY_IDS=""

TABLES="$DAILY $MONTHLY"
IDS="$DAILY_IDS $MONTHLY_IDS"

if [ $NONINTERACTIVE != 1 ]; then
    echo "Use zabbix;  SELECT 'Altering tables';" >$SQL
else
    echo "Use zabbix;" >$SQL
fi
cnt=0
for i in $TABLES; do
    if [ $NONINTERACTIVE != 1 ]; then
        echo "Altering table: $i"
        echo "SELECT '$i';" >>$SQL
    fi
    cnt=$((cnt+1))
    case $i in
        history_log)
            #echo "ALTER TABLE $i DROP KEY history_log_2;" >>$SQL
            #echo "ALTER TABLE $i ADD KEY history_log_2(itemid, id);" >>$SQL
            #echo "ALTER TABLE $i DROP PRIMARY KEY ;" >>$SQL
            #id=`echo $IDS | cut -f$cnt -d" "`
            #echo "ALTER TABLE $i ADD KEY ${i}id ($id);" >>$SQL
            ;;
        history_text)
            #echo "ALTER TABLE $i DROP KEY history_text_2;" >>$SQL
            #echo "ALTER TABLE $i ADD KEY history_text_2 (itemid, clock);" >>$SQL
            #echo "ALTER TABLE $i DROP PRIMARY KEY ;" >>$SQL
            #id=`echo $IDS | cut -f$cnt -d" "`
            #echo "ALTER TABLE $i ADD KEY ${i}id ($id);" >>$SQL
            ;;
    esac
done

echo -en "\n" >>$SQL
for i in $MONTHLY; do
    if [ $NONINTERACTIVE != 1 ]; then
        echo "Creating monthly partitions for table: $i"
        echo "SELECT '$i';" >>$SQL
    fi
    echo "ALTER TABLE $i PARTITION BY RANGE( clock ) (" >>$SQL
    for y in `seq $first_year $last_year`; do
        last_month=12
        [ $y -eq $last_year ] && last_month=$((cur_month+1))
        for m in `seq 1 $last_month`; do
            [ $m -lt 10 ] && m="0$m"
            ms=`date +"%Y-%m-01" -d "$m/01/$y +1 month"`
            pname="p${y}${m}"
            echo -n "PARTITION $pname  VALUES LESS THAN (UNIX_TIMESTAMP(\"$ms 00:00:00\"))" >>$SQL
            [ $m -ne $last_month -o $y -ne $last_year ] && echo -n "," >>$SQL
            echo -ne "\n" >>$SQL
        done
    done
    echo ");" >>$SQL
done

for i in $DAILY; do
    if [ $NONINTERACTIVE != 1 ]; then
        echo "Creating daily partitions for table: $i"
        echo "SELECT '$i';" >>$SQL
    fi
    echo "ALTER TABLE $i PARTITION BY RANGE( clock ) (" >>$SQL
    for d in `seq -$daily_history_min 2`; do
        ds=`date +"%Y-%m-%d" -d "$d day +1 day"`
        pname=`date +"%Y%m%d" -d "$d day"`
        echo -n "PARTITION p$pname  VALUES LESS THAN (UNIX_TIMESTAMP(\"$ds 00:00:00\"))" >>$SQL
        [ $d -ne 2 ] && echo -n "," >>$SQL
        echo -ne "\n" >>$SQL
    done
    echo ");" >>$SQL
done



###############################################################
if [ $NONINTERACTIVE != 1 ]; then
    cat >>$SQL <<_EOF_
SELECT "Installing procedures";
_EOF_
fi

cat >>$SQL <<_EOF_
/**************************************************************
  MySQL Auto Partitioning Procedure for Zabbix 1.8
  http://zabbixzone.com/zabbix/partitioning-tables/

  Author:  Ricardo Santos (rsantos at gmail.com)
  Version: 20110518
**************************************************************/
DELIMITER //
DROP PROCEDURE IF EXISTS zabbix.create_zabbix_partitions; //
CREATE PROCEDURE zabbix.create_zabbix_partitions ()
BEGIN
_EOF_

###############################################################

for i in $DAILY; do
    echo "    CALL zabbix.create_next_partitions(\"zabbix\",\"$i\");" >>$SQL
    echo "    CALL zabbix.drop_old_partitions(\"zabbix\",\"$i\");" >>$SQL
done
echo -en "\n" >>$SQL
for i in $MONTHLY; do
    echo "    CALL zabbix.create_next_monthly_partitions(\"zabbix\",\"$i\");" >>$SQL
    echo "    CALL zabbix.drop_old_monthly_partitions(\"zabbix\",\"$i\");" >>$SQL
done

###############################################################
cat >>$SQL <<_EOF_
END //

DROP PROCEDURE IF EXISTS zabbix.create_next_partitions; //
CREATE PROCEDURE zabbix.create_next_partitions (SCHEMANAME varchar(64), TABLENAME varchar(64))
BEGIN
    DECLARE NEXTCLOCK timestamp;
    DECLARE PARTITIONNAME varchar(16);
    DECLARE CLOCK int;
    SET @totaldays = 7;
    SET @i = 1;
    createloop: LOOP
        SET NEXTCLOCK = DATE_ADD(NOW(),INTERVAL @i DAY);
        SET PARTITIONNAME = DATE_FORMAT( NEXTCLOCK, 'p%Y%m%d' );
        SET CLOCK = UNIX_TIMESTAMP(DATE_FORMAT(DATE_ADD( NEXTCLOCK ,INTERVAL 1 DAY),'%Y-%m-%d 00:00:00'));
        CALL zabbix.create_partition( SCHEMANAME, TABLENAME, PARTITIONNAME, CLOCK );
        SET @i=@i+1;
        IF @i > @totaldays THEN
            LEAVE createloop;
        END IF;
    END LOOP;
END //


DROP PROCEDURE IF EXISTS zabbix.drop_old_partitions; //
CREATE PROCEDURE zabbix.drop_old_partitions (SCHEMANAME varchar(64), TABLENAME varchar(64))
BEGIN
    DECLARE OLDCLOCK timestamp;
    DECLARE PARTITIONNAME varchar(16);
    DECLARE CLOCK int;
    SET @mindays = $daily_history_min;
    SET @maxdays = @mindays+4;
    SET @i = @maxdays;
    droploop: LOOP
        SET OLDCLOCK = DATE_SUB(NOW(),INTERVAL @i DAY);
        SET PARTITIONNAME = DATE_FORMAT( OLDCLOCK, 'p%Y%m%d' );
        CALL zabbix.drop_partition( SCHEMANAME, TABLENAME, PARTITIONNAME );
        SET @i=@i-1;
        IF @i <= @mindays THEN
            LEAVE droploop;
        END IF;
    END LOOP;
END //

DROP PROCEDURE IF EXISTS zabbix.create_next_monthly_partitions; //
CREATE PROCEDURE zabbix.create_next_monthly_partitions (SCHEMANAME varchar(64), TABLENAME varchar(64))
BEGIN
    DECLARE NEXTCLOCK timestamp;
    DECLARE PARTITIONNAME varchar(16);
    DECLARE CLOCK int;
    SET @totalmonths = 3;
    SET @i = 1;
    createloop: LOOP
        SET NEXTCLOCK = DATE_ADD(NOW(),INTERVAL @i MONTH);
        SET PARTITIONNAME = DATE_FORMAT( NEXTCLOCK, 'p%Y%m' );
        SET CLOCK = UNIX_TIMESTAMP(DATE_FORMAT(DATE_ADD( NEXTCLOCK ,INTERVAL 1 MONTH),'%Y-%m-01 00:00:00'));
        CALL zabbix.create_partition( SCHEMANAME, TABLENAME, PARTITIONNAME, CLOCK );
        SET @i=@i+1;
        IF @i > @totalmonths THEN
            LEAVE createloop;
        END IF;
    END LOOP;
END //

DROP PROCEDURE IF EXISTS zabbix.drop_old_monthly_partitions; //
CREATE PROCEDURE zabbix.drop_old_monthly_partitions (SCHEMANAME varchar(64), TABLENAME varchar(64))
BEGIN
    DECLARE OLDCLOCK timestamp;
    DECLARE PARTITIONNAME varchar(16);
    DECLARE CLOCK int;
    SET @minmonths = $monthly_history_min;
    SET @maxmonths = @minmonths+24;
    SET @i = @maxmonths;
    droploop: LOOP
        SET OLDCLOCK = DATE_SUB(NOW(),INTERVAL @i MONTH);
        SET PARTITIONNAME = DATE_FORMAT( OLDCLOCK, 'p%Y%m' );
        CALL zabbix.drop_partition( SCHEMANAME, TABLENAME, PARTITIONNAME );
        SET @i=@i-1;
        IF @i <= @minmonths THEN
            LEAVE droploop;
        END IF;
    END LOOP;
END //

DROP PROCEDURE IF EXISTS zabbix.create_partition; //
CREATE PROCEDURE zabbix.create_partition (SCHEMANAME varchar(64), TABLENAME varchar(64), PARTITIONNAME varchar(64), CLOCK int)
BEGIN
    DECLARE RETROWS int;
    SELECT COUNT(1) INTO RETROWS
        FROM information_schema.partitions
        WHERE table_schema = SCHEMANAME AND table_name = TABLENAME AND partition_name = PARTITIONNAME;

    IF RETROWS = 0 THEN
        SELECT CONCAT( "create_partition(", SCHEMANAME, ",", TABLENAME, ",", PARTITIONNAME, ",", CLOCK, ")" ) AS msg;
             SET @sql = CONCAT( 'ALTER TABLE ', SCHEMANAME, '.', TABLENAME, 
                ' ADD PARTITION (PARTITION ', PARTITIONNAME, ' VALUES LESS THAN (', CLOCK, '));' );
        PREPARE STMT FROM @sql;
        EXECUTE STMT;
        DEALLOCATE PREPARE STMT;
    END IF;
END //

DROP PROCEDURE IF EXISTS zabbix.drop_partition; //
CREATE PROCEDURE zabbix.drop_partition (SCHEMANAME varchar(64), TABLENAME varchar(64), PARTITIONNAME varchar(64))
BEGIN
    DECLARE RETROWS int;
    SELECT COUNT(1) INTO RETROWS
        FROM information_schema.partitions
        WHERE table_schema = SCHEMANAME AND table_name = TABLENAME AND partition_name = PARTITIONNAME;

    IF RETROWS = 1 THEN
        SELECT CONCAT( "drop_partition(", SCHEMANAME, ",", TABLENAME, ",", PARTITIONNAME, ")" ) AS msg;
             SET @sql = CONCAT( 'ALTER TABLE ', SCHEMANAME, '.', TABLENAME,
                ' DROP PARTITION ', PARTITIONNAME, ';' );
        PREPARE STMT FROM @sql;
        EXECUTE STMT;
        DEALLOCATE PREPARE STMT;
    END IF;
END //
DELIMITER ;
_EOF_

if [ $SIMULATE = 1 ]; then
    exit 0
fi

if [ $NONINTERACTIVE = 1 ]; then
    yn='y'
else
    echo -e "\n\nReady to apply script to database, this may take a while.(Y/n): "
    read yn
fi
if [ "$yn" != "n" -a "$yn" != "N" ]; then
    mysql --skip-column-names -h ${DBHOST} -u ${DBUSER} -p${DBPASS} <$SQL
fi

conf=/etc/zabbix/zabbix_server.conf
if [ $NONINTERACTIVE = 1 ]; then
    yn='y'
else
    echo -e "\nIf Zabbix Version = 2.0 \nDo you want to update the /etc/zabbix/zabbix_server.conf"
    echo -n "to disable housekeeping (Y/n): "
    read yn
fi
if [ "$yn" != "n" -a "$yn" != "N" ]; then
    cp $conf ${conf}.bak
    sed  -i "s/^# DisableHousekeeping=0/DisableHousekeeping=1/" $conf
    sed  -i "s/^DisableHousekeeping=0/DisableHousekeeping=1/" $conf
    /etc/init.d/zabbix-server stop
    sleep 5
    /etc/init.d/zabbix-server start 2>&1 > /dev/null
fi

tmpfile=/tmp/cron$$
if [ $NONINTERACTIVE = 1 ]; then
    yn='y'
else
    echo -ne "\nDo you want to update the crontab (Y/n): "
    read yn
fi
if [ "$yn" != "n" -a "$yn" != "N" ]; then
    where=
    while [ "$where" = "" ]; do
        if [ $NONINTERACTIVE = 1 ]; then
            where='Y'
        else
            echo "The crontab entry can be either in /etc/cron.daily, or added"
            echo -e "to the crontab for root\n"
            echo -n "Do you want to add this to the /etc/cron.daily directory (Y/n): "
            read where
        fi
        [ "$where" = "" -o "$where" = "y" ] && where="Y"
        if [ "$where" != "y" -a "$where" != "Y" -a "$where" != "n" -a "$where" != "N" ]; then
            where=""
            echo "Response not recognized, please try again"
        fi
    done

    if [ $NONINTERACTIVE != 1 ]; then
        echo -en "\nEnter email of who should get the daily housekeeping reports: "
        read mailto
    fi
    [ "$mailto" = "" ] && mailto=$EMAIL
    mkdir -p $PATHTOCRON
    cat >$PATHTOCRON/housekeeping.sh <<_EOF_
#!/bin/bash

MAILTO=$mailto
tmpfile=/tmp/housekeeping\$\$

date >\$tmpfile
/usr/bin/mysql --skip-column-names -B -h localhost -u${DBUSER} -p${DBPASS} zabbix -e "CALL create_zabbix_partitions();" >>\$tmpfile 2>&1
$PATHTOMAILBIN -s "Zabbix MySql Partition Housekeeping" \$MAILTO <\$tmpfile
rm -f \$tmpfile
_EOF_
    chmod +x $PATHTOCRON/housekeeping.sh
    chown -R zabbix.zabbix /usr/local/zabbix
    if [ "$where" = "Y" ]; then
        cat >/etc/cron.daily/zabbixhousekeeping <<_EOF_
#!/bin/bash
$PATHTOCRON/housekeeping.sh
_EOF_
        chmod +x /etc/cron.daily/zabbixhousekeeping
    else
        crontab -l >$tmpfile
        cat >>$tmpfile <<_EOF_
0 0 * * *  $PATHTOCRON/housekeeping.sh
_EOF_
        crontab $tmpfile
        rm $tmpfile
    fi
fi

網上還有一些腳本，是手動執行一些mysql命令來分表，沒試過。

比如：https://www.cnblogs.com/w787815/p/8249303.html

3、執行分區腳本

備份數據庫：
[root@localhost ~]# mysqldump -uroot -p zabbix>/root/zabbix_$(date +%F).sql
停止數據庫：
[root@localhost ~]# systemctl stop zabbix-server
查看數據庫狀態：
[root@localhost ~]# systemctl status zabbix-server

執行腳本：

[root@localhost ~]# ./partitiontables_gt_zbx34.sh
Ready to partition tables.
Ready to update permissions of Zabbix user to create routines
#mysql的root帳號和密碼：
Enter root DB user: root
Enter root password: 123456
#備份數據庫的操作，已經手動備份：
Do you want to backup the database (recommended) (Y/n): n
Are you certain you have a backup (y/N): 
y
Ready to proceed:
Starting yearly partioning at: 2018
and ending at: 2018
With 30 days of daily history
Ready to proceed (Y/n): 
y
Altering table: history
Altering table: history_log
Altering table: history_str
Altering table: history_text
Altering table: history_uint
Altering table: trends
Altering table: trends_uint
Creating monthly partitions for table: trends
Creating monthly partitions for table: trends_uint
Creating daily partitions for table: history
Creating daily partitions for table: history_log
Creating daily partitions for table: history_str
Creating daily partitions for table: history_text
Creating daily partitions for table: history_uint
Ready to apply script to database, this may take a while.(Y/n): 
y
Altering tables
history
history_log
history_str
history_text
history_uint
trends
trends_uint
trends
trends_uint
history
history_log
history_str
history_text
history_uint
Installing procedures
If Zabbix Version = 2.0 
Do you want to update the /etc/zabbix/zabbix_server.conf
to disable housekeeping (Y/n): n
Do you want to update the crontab (Y/n): y
The crontab entry can be either in /etc/cron.daily, or added
to the crontab for root
Do you want to add this to the /etc/cron.daily directory (Y/n): y
#設置清理報告通知郵箱：
Enter email of who should get the daily housekeeping reports: [email protected]

如果您順序運行到這裏，那麽恭喜你成功了。

4、確認定時任務
上面的腳本會在/etc/cron.daily目錄下生成一個名稱為zabbixhousekeeping的腳本：

[root@localhost ~]# cat /etc/cron.daily/zabbixhousekeeping 
#!/bin/bash
/usr/local/zabbix/cron.d/housekeeping.sh

上面的腳本會在/usr/local/zabbix/cron.d/目錄下生成一個名稱為housekeeping.sh腳本。

[root@localhost ~]# cat /usr/local/zabbix/cron.d/housekeeping.sh 
#!/bin/bash
[email protected]
tmpfile=/tmp/housekeeping$$
date >$tmpfile
/usr/bin/mysql --skip-column-names -B -h localhost -uzabbix -pzabbix zabbix -e "CALL create_zabbix_partitions();" >>$tmpfile 2>&1
/usr/bin/mail -s "Zabbix MySql Partition Housekeeping" $MAILTO <$tmpfile
rm -f $tmpfile

這樣的話，每天淩晨大概3點左右會執行，並且用mail程序發送執行結果到上面的你指定的郵箱。可以看到刪除過期的數據分區表，並建立新的分區表。

5、確認腳本執行效果
執行以下SQL語句查看histroy表的分區情況：

MariaDB [(none)]> use zabbix;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [zabbix]> select 
    ->   partition_name part,  
    ->   partition_expression expr,  
    ->   partition_description descr,  
    ->   table_rows  
    -> from information_schema.partitions  where 
    ->   table_schema = schema()  
    ->   and table_name='history';
+-----------+--------+------------+------------+
| part      | expr   | descr      | table_rows |
+-----------+--------+------------+------------+
| p20180219 |  clock | 1519056000 |       3757 |
| p20180220 |  clock | 1519142400 |       6455 |
| p20180221 |  clock | 1519228800 |       6887 |
| p20180222 |  clock | 1519315200 |       6455 |
| p20180223 |  clock | 1519401600 |       6887 |
| p20180224 |  clock | 1519488000 |       6574 |
| p20180225 |  clock | 1519574400 |       6455 |
| p20180226 |  clock | 1519660800 |       6887 |
| p20180227 |  clock | 1519747200 |       6455 |
| p20180228 |  clock | 1519833600 |       6887 |
| p20180301 |  clock | 1519920000 |       6887 |
| p20180302 |  clock | 1520006400 |       6574 |
| p20180303 |  clock | 1520092800 |       5720 |
| p20180304 |  clock | 1520179200 |       6161 |
| p20180305 |  clock | 1520265600 |       6574 |
| p20180306 |  clock | 1520352000 |       6513 |
| p20180307 |  clock | 1520438400 |       6574 |
| p20180308 |  clock | 1520524800 |       6469 |
| p20180309 |  clock | 1520611200 |       6574 |
| p20180310 |  clock | 1520697600 |       6030 |
| p20180311 |  clock | 1520784000 |       6161 |
| p20180312 |  clock | 1520870400 |       6455 |
| p20180313 |  clock | 1520956800 |       6887 |
| p20180314 |  clock | 1521043200 |     234759 |
| p20180315 |  clock | 1521129600 |     442278 |
| p20180316 |  clock | 1521216000 |     441965 |
| p20180317 |  clock | 1521302400 |     443530 |
| p20180318 |  clock | 1521388800 |     441965 |
| p20180319 |  clock | 1521475200 |     441652 |
| p20180320 |  clock | 1521561600 |    2345005 |
| p20180321 |  clock | 1521648000 |   12088382 |
| p20180322 |  clock | 1521734400 |          0 |
| p20180323 |  clock | 1521820800 |          0 |
+-----------+--------+------------+------------+
33 rows in set (0.83 sec)

分區表成功，啟動zabbix-server服務。

6、取消housekeeper清理

技術分享圖片

zabbix3.2分區表配置

zabbix3.2分區表配置 mysql分區表配置添加了將近1000臺主機，數據庫中的歷史數據越來越多了。出現了zabbix自帶的housekeeper清理歷史數據的時候，造成查數據慢並且觸發了很多報警信息（比如Zabbix housekeeper processes more than 75% b

zabbix3.2分區表配置

zabbix3.2分區表配置

pg 10多級分區表（range_list）配置查詢

在STM32上實現NTFS之5：GPT分區表的C語言實現（2）GPT實現以及統一方式讀取磁盤分區

oracle 分區表

Oracle分區表的move操作

分區表(主對象、子對象)歷史統計信息

Oracle中分區表中表空間屬性

hive分區表實踐

分區表 fstab

linux中生成考核用的GPT分區表結構修復案例

mysql分區表的原理和優缺點

給虛擬機添加新硬盤並分區，fdisk查看分區，分區，重新讀取分區表信息partprobe，格式化，掛載，查看分區掛載信息

mysql分區表

MySQL-分區表-1

創建分區表（按照年份分區，自動新增分區）

mssqlserver分區表的左值與右值

SQL Server 2008如何創建分區表，並壓縮數據庫空間

SQL Server 2005中的分區表（五）：添加一個分區

SqlServer 2005 將已存在大量數據的表更改為分區表

SQL Server 2005中的分區表（三）：將普通表轉換成分區表

zabbix3.2分區表配置

相關推薦