logminer使用測試庫進行挖掘分析，10.2.0.5

阿新 • • 發佈：2019-01-13

上一篇測試是在dg環境進行測試挖掘，但是如果客戶存在一個測試庫，那樣使用日誌挖掘的影響性更小。本篇進行測試分析。

測試環境介紹：

oracle linux 5.6，vmware虛擬機器，安裝兩套單例項10.2.0.5資料庫，一套模擬生產環境進行dml操作，另一套模擬測試環境進行日誌相關挖掘。

測試流程說明：

1.生產環境，模擬dml操作，一個表A，產生一個delete 1行記錄，執行兩次，表B，執行一次delete操作， 30000條記錄，日誌切換（歸檔模式下），再次多切換幾次日誌

2.目標端，使用logminer 進行挖掘相關日誌，最終得出相關分析。

一、生產環境日誌模擬

SQL> show parameter name
NAME        VALUE
--------------------
db_name    test1

SQL> archive log list
Database log mode Archive Mode
Automatic archival Enabled
Archive destination /u02/app/oracle/arch


SQL> create user test1 identified by test1;
SQL> create user test2 identified by test2;
SQL> grant dba to test1,test2;

SQL> conn test1/test1

SQL> create table a as select * from scott.emp;

SQL> delete a where rownum=1;

SQL> commit;

SQL> delete a where empno=7788;

SQL> commit;

SQL> alter system switch logfile;

SQL> conn test2/test2
SQL> create table b as select * from dba_objects;

SQL> insert into b select * from b;

SQL> insert into b select * from b

SQL> commit;

SQL> delete b where rownum<100000;

SQL> commit;

SQL> alter system archive log current;

SQL> select DEST_ID,THREAD# ,SEQUENCE#,COMPLETION_TIME,NAME from v$archived_log where COMPLETION_TIME >sysdate-20/1440;

DEST_ID THREAD# SEQUENCE# COMPLETION_TIME NAME
---------- ---------- ---------- ------------------- -------------------------------------------------------
1 1 1 2019-01-12 12:02:42 /u02/app/oracle/arch/1_1_993126050.arc
1 1 2 2019-01-12 12:14:46 /u02/app/oracle/arch/1_2_993126050.arc
1 1 3 2019-01-12 12:16:46 /u02/app/oracle/arch/1_3_993126050.arc

二、測試庫使用Logminer

1）logminer前提準備

新增儲存過程

@?/rdbms/admin/dbmslmd.sql
@?/rdbms/admin/dbmslm.sql
@?/rdbms/admin/dbmslms.sql
@?/rdbms/admin/prvtlm.plb

開啟最小補充日誌

select SUPPLEMENTAL_LOG_DATA_MIN,SUPPLEMENTAL_LOG_DATA_PK,SUPPLEMENTAL_LOG_DATA_UI from v$database;

SQL> alter database add supplemental log data;

Database altered.

SQL> select SUPPLEMENTAL_LOG_DATA_MIN,SUPPLEMENTAL_LOG_DATA_PK,SUPPLEMENTAL_LOG_DATA_UI from v$database;

SUPPLEME SUP SUP
-------- --- ---
YES NO NO

2）日誌挖掘，在無資料字典的情況下，輸出內容

SQL> exec dbms_logmnr_d.build(options => dbms_logmnr_d.STORE_IN_REDO_LOGS);

SQL> exec dbms_logmnr.add_logfile('/u02/app/oracle/arch/1_1_993126050.arc',dbms_logmnr.new);

SQL> exec dbms_logmnr.add_logfile('/u02/app/oracle/arch/1_2_993126050.arc',dbms_logmnr.addfile);

使用online 資料字典進行翻譯，報錯： dbid不同

SQL> execute dbms_logmnr.start_logmnr(options=>dbms_logmnr.dict_from_online_catalog);
BEGIN dbms_logmnr.start_logmnr(options=>dbms_logmnr.dict_from_online_catalog); END;

*
ERROR at line 1:
ORA-01295: DB_ID mismatch between dictionary USE_ONLINE_CATALOG and logfiles
ORA-06512: at "SYS.DBMS_LOGMNR", line 58
ORA-06512: at line 1

SQL> exec DBMS_LOGMNR.START_LOGMNR(DictFileName=>'',Options=>0);

SQL> select sql_redo,sql_undo from V$LOGMNR_CONTENTS where sql_redo like '%insert into%';

SQL_REDO SQL_UNDO
--------------------------------------------------------------------------------
insert into "UNKNOWN"."OBJ# 8869"("COL 1","COL 2","COL 3","COL 4","COL 5") value
s (HEXTORAW('c2020d'),HEXTORAW('c104'),HEXTORAW('7465737431584442'),HEXTORAW('c1
06'),HEXTORAW('c102'));
delete from "UNKNOWN"."OBJ# 8869" where "COL 1" = HEXTORAW('c2020d') and "COL 2"
= HEXTORAW('c104') and "COL 3" = HEXTORAW('7465737431584442') and "COL 4" = HEX
TORAW('c106') and "COL 5" = HEXTORAW('c102') and ROWID = 'AAACKlAABAAAFRSAAA';

在使用異機使用logminer挖掘，在沒有資料字典的情況下，挖掘出來的只能是obj#，無法獲取物件名稱

select * from (
select rownum,username,SEG_OWNER,SEG_NAME,seg_type_name,OPERATION,a from (
select username,SEG_OWNER,SEG_NAME,seg_type_name,OPERATION,count(*) a
from V$LOGMNR_CONTENTS where OPERATION in('UPDATE','DELETE','INSERT') group by username,SEG_OWNER,SEG_NAME,seg_type_name,
OPERATION order by 6 desc)) where rownum<20

M USERNAME SEG_OWNER SEG_NAME SEG_TYPE_NAME OPERATION A
- ---------- --------------- --------------- --------------- -------------------------------- ----------
1 UNKNOWN UNKNOWN OBJ# 51696 UNKNOWN INSERT 5925
2 UNKNOWN UNKNOWN OBJ# 51724 UNKNOWN INSERT 5820
3 UNKNOWN UNKNOWN OBJ# 51728 UNKNOWN INSERT 3990
4 UNKNOWN UNKNOWN OBJ# 9115 UNKNOWN INSERT 3240

https://blog.csdn.net/cuiyan1982/article/details/80333013

3）生產環境，修改引數使用檔案儲存資料字典，資料字典檔案拷貝至測試庫進行註冊使用。

alter system set utl_file_dir=/abc scope=spfile; 生產環境需要重啟，代價太高

startup force --實際環境不能這麼幹

SQL> exec dbms_logmnr_d.build(dictionary_filename=>'ar1.dic',dictionary_location=>'/abc',options=>dbms_logmnr_d.STORE_IN_FLAT_FILE);

測試環境，使用該資料字典，進行解析。

SQL> exec dbms_logmnr_d.build(options => dbms_logmnr_d.STORE_IN_REDO_LOGS);

SQL> exec dbms_logmnr.add_logfile('/u02/app/oracle/arch/1_1_993126050.arc',dbms_logmnr.new);

SQL> exec dbms_logmnr.add_logfile('/u02/app/oracle/arch/1_2_993126050.arc',dbms_logmnr.addfile);

SQL> exec DBMS_LOGMNR.START_LOGMNR(DictFileName=>'/abc/ar1.dic',Options=>0);

ROWNUM USERNAME SEG_OWNER SEG_NAME SEG_TYPE_NAME OPERATION A
---------- ---------- ---------- -------------------- -------------------- ---------- ----------
1 UNKNOWN SYS WRH$_LATCH,WRH$_LATC TABPART INSERT 5925
H_1370159887_0

2 UNKNOWN SYS WRH$_SYSSTAT,WRH$_SY TABPART INSERT 5820
SSTA_1370159887_0

3 UNKNOWN SYS WRH$_PARAMETER,WRH$_ TABPART INSERT 3990
PARAME_1370159887_0

#查詢資料字典檔案（oracle根據資料字典，進行解析obj，轉換為我們熟悉的使用者名稱，表物件名稱等）

CREATE_TABLE DICTIONARY_TABLE ( DB_NAME VARCHAR2(9), DB_ID NUMBER(20), DB_CREATED VARCHAR2(20), DB_DICT_CREATED VARCHAR2(20), DB_RESETLOGS_CHANGE# NUMBER(22
), DB_RESETLOGS_TIME VARCHAR2(20), DB_VERSION_TIME VARCHAR2(20), DB_REDO_TYPE_ID VARCHAR2(8), DB_REDO_RELEASE VARCHAR2(60), DB_CHARACTER_SET VARCHAR2(30), D
B_VERSION VARCHAR2(64), DB_STATUS VARCHAR2(64), DB_DICT_MAXOBJECTS NUMBER(22), DB_DICT_OBJECTCOUNT NUMBER(22), DB_DICT_SCN NUMBER(22), DB_THREAD_MAP RAW(8),
DB_TXN_SCNBAS NUMBER(22), DB_TXN_SCNWRP NUMBER(22));

INSERT_INTO DICTIONARY_TABLE VALUES ('TEST1',1370159887,'11/25/2018 09:58:39','01/12/2019 13:56:03',420491,'11/25/2018 12:20:50','11/25/2018 11:36:00','',''
,'AL32UTF8','10.2.0.5.0','Production',53160,50195,493175,,493793,0);

CREATE_TABLE OBJ$_TABLE (OBJ# NUMBER(22), DATAOBJ# NUMBER(22), OWNER# NUMBER(22), NAME VARCHAR2(30), NAMESPACE NUMBER(22), SUBNAME VARCHAR2(30), TYPE# NUMBE
R(22), CTIME DATE, MTIME DATE, STIME DATE, STATUS NUMBER(22), REMOTEOWNER VARCHAR2(30), LINKNAME VARCHAR2(128), FLAGS NUMBER(22), OID$ RAW(16), SPARE1 NUMBE
R(22), SPARE2 NUMBER(22), SPARE3 NUMBER(22), SPARE4 VARCHAR2(1000), SPARE5 VARCHAR2(1000), SPARE6 DATE );

INSERT_INTO OBJ$_TABLE VALUES (20,2,0,'ICOL$',1,'',2,to_date('04/20/2010 08:24:28', 'MM/DD/YYYY HH24:MI:SS'),to_date('04/20/2010 08:32:25', 'MM/DD/YYYY HH24
:MI:SS'),to_date('04/20/2010 08:24:28', 'MM/DD/YYYY HH24:MI:SS'),1,'','',0,,0,1,,'','', );

4）生產環境，將資料字典，寫入Online redo檔案中，切換歸檔，拷貝至測試庫，註冊後使用。

實際環境中，根本不允許生產環境隨便重啟庫修改引數檔案，因此線上操作更可取。

SQL> alter database add supplemental log data; --需要開啟最小補充日誌

SQL> exec DBMS_LOGMNR_D.BUILD(dictionary_filename=>NULL,dictionary_location=>NULL,options=>dbms_logmnr_d.STORE_IN_REDO_LOGS);

col name format a90
set linesize 150
select name,ARCHIVED,DICTIONARY_BEGIN,DICTIONARY_END from v$archived_log where name like '%.arc';

NAME ARC DIC DIC
--------------------------------------------- --- --- ---

/u02/app/oracle/arch/1_9_993126050.arc YES YES YES

選擇將此歸檔檔案進行copy，無需手工切換，因為執行exec寫入redo，會自動切換產生歸檔日誌

測試環境，匯入這個歸檔日誌：

SQL> exec dbms_logmnr_d.build(options => dbms_logmnr_d.STORE_IN_REDO_LOGS);