1. 程式人生 > >ORACLE 11G RAC ASM磁盤組全部丟失後的恢復

ORACLE 11G RAC ASM磁盤組全部丟失後的恢復

實例 ice mat dns 禁用 buffers bit allocated event

一、環境描述
(1)Oracle 11.2.0.3 RAC ON Oracle Linux 6 x86_64,只有一個ASM外部冗余磁盤組——DATA;
(2)OCR,VOTEDISK,DATAFILE,CONTROLFILE,SPFILE全部位於這個磁盤組上;
二、故障描述
(1)存儲故障導致ASM磁盤丟失。
(2)CRS因為OCR和VOTEDISK的丟失,除了OHAS還聯機外,CLUSTERWARE服務都已經停止。
三、備份情況
(1)RMAN備份:包括controlfile,database,spfile,archivelog,
(2)OCR備份:沒有進行過人工備份,在$CRS_HOME/cdata目錄下有CRS自動備份文件。

四、操作步驟
說明:準使用CRS自動備份的文件恢復OCR,使用RMAN備份來恢復數據庫;準備恢復數據的同時,調整ASM磁盤組,將OCR,VOTEDISK同數據庫文件分開存放。
4.1 恢復OCR和VOTEDISK
(1) 在所有RAC節點上停止CRS服務
[root@rac1 ~]# crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘rac1‘
CRS-2673: Attempting to stop ‘ora.mdnsd‘ on ‘rac1‘

CRS-2673: Attempting to stop ‘ora.crf‘ on ‘rac1‘
CRS-2677: Stop of ‘ora.mdnsd‘ on ‘rac1‘ succeeded
CRS-2677: Stop of ‘ora.crf‘ on ‘rac1‘ succeeded
CRS-2673: Attempting to stop ‘ora.gipcd‘ on ‘rac1‘
CRS-2677: Stop of ‘ora.gipcd‘ on ‘rac1‘ succeeded
CRS-2673: Attempting to stop ‘ora.gpnpd‘ on ‘rac1‘
CRS-2677: Stop of ‘ora.gpnpd‘ on ‘rac1‘ succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘rac1‘ has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac2 ~]# crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘rac2‘
CRS-2673: Attempting to stop ‘ora.mdnsd‘ on ‘rac2‘
CRS-2673: Attempting to stop ‘ora.crf‘ on ‘rac2‘
CRS-2677: Stop of ‘ora.mdnsd‘ on ‘rac2‘ succeeded
CRS-2677: Stop of ‘ora.crf‘ on ‘rac2‘ succeeded
CRS-2673: Attempting to stop ‘ora.gipcd‘ on ‘rac2‘
CRS-2677: Stop of ‘ora.gipcd‘ on ‘rac2‘ succeeded
CRS-2673: Attempting to stop ‘ora.gpnpd‘ on ‘rac2‘
CRS-2677: Stop of ‘ora.gpnpd‘ on ‘rac2‘ succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘rac2‘ has completed
CRS-4133: Oracle High Availability Services has been stopped.
(2) 在一個節點上以NOCRS方式啟動CRS,此操作會啟動ASM實例。
[root@rac1 ~]# crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start ‘ora.mdnsd‘ on ‘rac1‘
CRS-2676: Start of ‘ora.mdnsd‘ on ‘rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.gpnpd‘ on ‘rac1‘
CRS-2676: Start of ‘ora.gpnpd‘ on ‘rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.cssdmonitor‘ on ‘rac1‘
CRS-2672: Attempting to start ‘ora.gipcd‘ on ‘rac1‘
CRS-2676: Start of ‘ora.cssdmonitor‘ on ‘rac1‘ succeeded
CRS-2676: Start of ‘ora.gipcd‘ on ‘rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.cssd‘ on ‘rac1‘
CRS-2672: Attempting to start ‘ora.diskmon‘ on ‘rac1‘
CRS-2676: Start of ‘ora.diskmon‘ on ‘rac1‘ succeeded
CRS-2676: Start of ‘ora.cssd‘ on ‘rac1‘ succeeded
CRS-2679: Attempting to clean ‘ora.cluster_interconnect.haip‘ on ‘rac1‘
CRS-2672: Attempting to start ‘ora.ctssd‘ on ‘rac1‘
CRS-2681: Clean of ‘ora.cluster_interconnect.haip‘ on ‘rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip‘ on ‘rac1‘
CRS-2676: Start of ‘ora.ctssd‘ on ‘rac1‘ succeeded
CRS-2676: Start of ‘ora.cluster_interconnect.haip‘ on ‘rac1‘ succeeded
CRS-2672: Attempting to start ‘ora.asm‘ on ‘rac1‘
CRS-2676: Start of ‘ora.asm‘ on ‘rac1‘ succeeded

(3) 新添加了三塊磁盤,已經使用UDEV進行了綁定,查看磁盤狀態。

[root@rac1 ~]# su - grid
[grid@rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Fri Jul 5 17:41:49 2013

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> select group_number group#, disk_number disk#, OS_MB, state, path, header_status from v$asm_disk order by 1,2;

GROUP#      DISK#      OS_MB STATE      PATH                 HEADER_STATUS

     0          0       1024 NORMAL     /dev/asm-diskc       CANDIDATE
     0          1       5120 NORMAL     /dev/asm-diskd       CANDIDATE
     0          2      20480 NORMAL     /dev/asm-diskb       CANDIDATE

(4) 創建三個磁盤組,SYSTEMDG給CRS使用,用於存放OCR,VOTEDISK和ASM實例的SPFILE。其余兩個給ORACLE使用,DATADG用於存放datafile,controlfile,redolog,spfile;ARCLOGDG存放archivelog。

SQL> create diskgroup SYSTEMDG external redundancy
2 disk ‘/dev/asm-diskc‘
3 ATTRIBUTE ‘compatible.rdbms‘ = ‘11.2‘,‘compatible.asm‘ = ‘11.2‘;

Diskgroup created.

SQL> create diskgroup DATADG external redundancy
2 disk ‘/dev/asm-diskb‘
3 ATTRIBUTE ‘compatible.rdbms‘ = ‘11.2‘,‘compatible.asm‘ = ‘11.2‘;

Diskgroup created.

SQL> create diskgroup ARCLOGDG external redundancy
2 disk ‘/dev/asm-diskd‘
3 ATTRIBUTE ‘compatible.rdbms‘ = ‘11.2‘,‘compatible.asm‘ = ‘11.2‘;

Diskgroup created.

(5) 準備恢復OCR和VOTEDISK,/etc/oracle/ocr.loc中記錄了OCR路徑,修改ocrconfig_loc的值,以便將OCR恢復到新的磁盤組中。

[root@rac1 ~]# more /etc/oracle/ocr.loc
ocrconfig_loc=+DATA
local_only=FALSE

[root@rac1 ~]# vi /etc/oracle/ocr.loc

ocrconfig_loc=+SYSTEMDG
local_only=FALSE

(6) 恢復OCR

[root@rac1 ~]# ocrconfig -showbackup
PROT-26: Oracle Cluster Registry backup locations were retrieved from a local copy

rac1 2013/07/05 12:30:00 /u01/app/11.2.0/grid/cdata/rac-cluster/backup00.ocr

rac1 2013/07/05 08:30:00 /u01/app/11.2.0/grid/cdata/rac-cluster/backup01.ocr

rac1 2013/07/05 04:30:00 /u01/app/11.2.0/grid/cdata/rac-cluster/backup02.ocr

rac1 2013/07/05 00:29:59 /u01/app/11.2.0/grid/cdata/rac-cluster/day.ocr

rac1 2013/07/05 00:29:59 /u01/app/11.2.0/grid/cdata/rac-cluster/week.ocr
PROT-25: Manual backups for the Oracle Cluster Registry are not available

[root@rac1 ~]# ocrconfig -restore /u01/app/11.2.0/grid/cdata/rac-cluster/backup00.ocr
[root@rac1 ~]#
[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2840
Available space (kbytes) : 259280
ID : 59415097
Device/File Name : +SYSTEMDG
Device/File integrity check succeeded

                                Device/File not configured

                                Device/File not configured

                                Device/File not configured

                                Device/File not configured

     Cluster registry integrity check succeeded

     Logical corruption check succeeded

(7) 創建VOTEDISK

[root@rac1 ~]# crsctl replace votedisk +SYSTEMDG
CRS-4602: Failed 27 to add voting file afb0ca0f35684f1abfd43d5ec2dc1123.
Failed to replace voting disk group with +SYSTEMDG.
CRS-4000: Command Replace failed, or completed with errors.

以上報錯是因為使用UDEV綁定ASM磁盤時需要更改默認磁盤搜索路徑為/dev/asm*,修改ASM磁盤搜索路徑

[root@rac1 ~]# su - grid
[grid@rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Fri Jul 5 19:03:25 2013

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> show parameter asm_diskstring

NAME TYPE VALUE


asm_diskstring string
SQL>
SQL>
SQL> alter system set asm_diskstring = ‘/dev/asm*‘;

System altered.

SQL> create spfile from memory;
create spfile from memory
*
ERROR at line 1:
ORA-00349: failure obtaining block size for
‘+DATA/rac-cluster/asmparameterfile/registry.253.819922365‘
ORA-15001: diskgroup "DATA" does not exist or is not mounted

SQL> create spfile=‘+SYSTEMDG‘ from memory;

File created.

SQL> startup force mount;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started

Total System Global Area 283930624 bytes
Fixed Size 2227664 bytes
Variable Size 256537136 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted

在次創建VOTEDISK,成功。

[root@rac1 init]# crsctl replace votedisk +SYSTEMDG
Successful addition of voting disk 8ebb7a63accb4fa8bfa7ab65df7a8c8a.
Successfully replaced voting disk group with +SYSTEMDG.
CRS-4266: Voting file(s) successfully replaced

(8) OCR和VOTEDISK都恢復完成後,重啟CRS到正常模式。

[root@rac1 ~]# crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘rac1‘
CRS-2673: Attempting to stop ‘ora.mdnsd‘ on ‘rac1‘
CRS-2673: Attempting to stop ‘ora.ctssd‘ on ‘rac1‘
CRS-2673: Attempting to stop ‘ora.asm‘ on ‘rac1‘
CRS-2677: Stop of ‘ora.mdnsd‘ on ‘rac1‘ succeeded
CRS-2677: Stop of ‘ora.asm‘ on ‘rac1‘ succeeded
CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip‘ on ‘rac1‘
CRS-2677: Stop of ‘ora.ctssd‘ on ‘rac1‘ succeeded
CRS-2677: Stop of ‘ora.cluster_interconnect.haip‘ on ‘rac1‘ succeeded
CRS-2673: Attempting to stop ‘ora.cssd‘ on ‘rac1‘
CRS-2677: Stop of ‘ora.cssd‘ on ‘rac1‘ succeeded
CRS-2673: Attempting to stop ‘ora.gipcd‘ on ‘rac1‘
CRS-2677: Stop of ‘ora.gipcd‘ on ‘rac1‘ succeeded
CRS-2673: Attempting to stop ‘ora.gpnpd‘ on ‘rac1‘
CRS-2677: Stop of ‘ora.gpnpd‘ on ‘rac1‘ succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘rac1‘ has completed
CRS-4133: Oracle High Availability Services has been stopped.

[root@rac1 ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.

[root@rac1 ~]# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@rac1 ~]#

4.2 修改CRS註冊表中相關配置信息

(1) 掛載新的ASM磁盤組

[grid@rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 00:16:05 2013

Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL>
SQL> select name,state from v$asm_diskgroup;

NAME STATE


SYSTEMDG MOUNTED
ARCLOGDG DISMOUNTED
DATADG DISMOUNTED

SQL> alter diskgroup ARCLOGDG,DATADG mount;

Diskgroup altered.

(2) 更改CRS配置文件中數據庫的磁盤組為DATADG和ARCLOGDG

[root@rac1 ~]# srvctl modify database -d csdb -a "DATADG,ARCLOGDG"

(3) 禁用並刪除原來的磁盤組DATA

[root@rac1 ~]# srvctl disable diskgroup -g DATA
[root@rac1 ~]# srvctl remove diskgroup -g DATA

[root@rac1 rac-cluster]# crs_stat -t -v
Name Type R/RA F/FT Target State Host

ora....OGDG.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1
ora.DATADG.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1
ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac1
ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1
ora.csdb.db ora....se.type 0/2 0/1 ONLINE OFFLINE
ora.cvu ora.cvu.type 0/5 0/0 ONLINE ONLINE rac1
ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE
ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type 0/1 0/2 ONLINE ONLINE rac1
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1
ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1
ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE rac1
ora.rac1.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1
ora.rac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1
ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac1

(4) 在OCR註冊表中修改Oracle數據庫參數文件的位置

[root@rac1 ~]# srvctl modify database -d csdb -p +DATADG/csdb/spfilecsdb.ora

4.3 恢復數據庫

(1) 查看備份文件路徑和名稱

[root@rac1 ~]# su - oracle
[oracle@rac1 ~]$
[oracle@rac1 ~]$ cd /u01/app/oracle/backup
[oracle@rac1 backup]$ ll
total 221796
-rw-r----- 1 oracle asmadmin 5357568 Jul 5 15:19 arc_819991156_9.bk
-rw-r----- 1 oracle asmadmin 2560 Jul 5 15:19 arc_819991158_11.bk
-rw-r----- 1 oracle asmadmin 203104256 Jul 5 15:18 CSDB_819991120_5.bk
-rw-r----- 1 oracle asmadmin 18546688 Jul 5 15:19 ctl_file_0coe04jq_1_1_20130705.ctl
-rw-r----- 1 oracle asmadmin 98304 Jul 5 15:19 spfile_0doe04js_1_1_20130705
[oracle@rac1 backup]$

(2) 創建一個基本的啟動參數文件,以便啟動數據庫到nomout狀態恢復spfile

[oracle@rac1 ~]$ touch /u01/app/oracle/backup/init.ora
[oracle@rac1 ~]$ vi /u01/app/oracle/backup/init.ora

.db_name=‘csdb‘
.remote_login_passwordfile=‘exclusive‘

(3) 使用剛創建的參數文件將數據庫啟動到nomount狀態

[oracle@rac1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 13:56:06 2013

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup nomount pfile=‘/u01/app/oracle/backup/init.ora‘;
ORACLE instance started.

Total System Global Area 238034944 bytes
Fixed Size 2227136 bytes
Variable Size 180356160 bytes
Database Buffers 50331648 bytes
Redo Buffers 5120000 bytes
SQL>

(4) 使用RMAN恢復SPFILE到ASM磁盤組DATADG

[oracle@rac1 ~]$ rman target /

Recovery Manager: Release 11.2.0.3.0 - Production on Sat Jul 6 13:59:26 2013

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

connected to target database: CSDB (not mounted)

RMAN> restore spfile to ‘+DATADG/csdb/spfilecsdb.ora‘ from ‘/u01/app/oracle/backup/spfile_0doe04js_1_1_20130705‘;

Starting restore at 06-JUL-13
using channel ORA_DISK_1

channel ORA_DISK_1: restoring spfile from AUTOBACKUP /u01/app/oracle/backup/spfile_0doe04js_1_1_20130705
channel ORA_DISK_1: SPFILE restore from AUTOBACKUP complete
Finished restore at 06-JUL-13

(5) 使用恢復後spfile啟動數據庫,並修改control_files,db_recovery_file_dest,log_archive_dest等存在舊路徑的參數值。

[oracle@rac1 ~]$ vi $ORACLE_HOME/dbs/initcsdb1.ora

SPFILE=‘+DATADG/csdb/spfilecsdb.ora‘

[oracle@rac1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 14:11:58 2013

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options

SQL> startup nomount force
ORACLE instance started.

Total System Global Area 1653518336 bytes
Fixed Size 2228904 bytes
Variable Size 1073745240 bytes
Database Buffers 570425344 bytes
Redo Buffers 7118848 bytes
SQL>
SQL> show parameter control_files

NAME TYPE VALUE


control_files string +DATA/csdb/control01.ctl, +DAT
A/csdb/control02.ctl
SQL>
SQL> alter system set control_files=‘+DATADG/csdb/control01.ctl‘,‘+DATADG/csdb/control02.ctl‘ scope=spfile

System altered.

SQL> alter system set db_recovery_file_dest=‘+DATADG‘ scope=spfile;

System altered.

SQL> alter system set log_archive_dest_1=‘LOCATION=+ARCLOGDG‘ scope=spfile;

System altered.

SQL> startup force nomount;
ORACLE instance started.

Total System Global Area 1653518336 bytes
Fixed Size 2228904 bytes
Variable Size 1073745240 bytes
Database Buffers 570425344 bytes
Redo Buffers 7118848 bytes

(6) 查看數據庫的DBID

[oracle@rac1 ~]$ strings /u01/app/oracle/backup/CSDB_819991120_5.bk | grep MAXVALUE,

返回的值類似下面的例子,其中那一竄數字即為DBID。
...
MAXVALUE, MAXVALUE!
3042905279, MAXVALUE,
3042905279, MAXVALUE,
...

(7) 恢復控制文件到新的ASM磁盤組DATADG

[oracle@rac1 ~]$ rman target /

Recovery Manager: Release 11.2.0.3.0 - Production on Sat Jul 6 14:28:28 2013

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

connected to target database: CSDB (not mounted)

RMAN> set dbid=3042905279

executing command: SET DBID

RMAN> restore controlfile from ‘/u01/app/oracle/backup/ctl_file_0coe04jq_1_1_20130705.ctl‘;

Starting restore at 06-JUL-13
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=18 instance=csdb1 device type=DISK

channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
output file name=+DATADG/csdb/control01.ctl
output file name=+DATADG/csdb/control02.ctl
Finished restore at 06-JUL-13

(8) 進入SQLPLUS,查看舊數據文件信息

[oracle@rac1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 14:41:12 2013

Copyright (c) 1982, 2011, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> alter database mount;

Database altered.
SQL> col name format a50
SQL> select file#,name from v$datafile;

 FILE# NAME

     1 +DATA/csdb/datafile/system.260.819979847
     2 +DATA/csdb/datafile/sysaux.261.819979871
     3 +DATA/csdb/datafile/undotbs1.262.819979889
     4 +DATA/csdb/datafile/undotbs2.264.819979905
     5 +DATA/csdb/datafile/users.265.819979913

(9) 使用RMAN恢復數據庫

RMAN> run{
2> set newname for datafile 1 to ‘+DATADG/csdb/datafile/system.260.819979847‘;
3> set newname for datafile 2 to ‘+DATADG/csdb/datafile/sysaux.261.819979871‘;
4> set newname for datafile 3 to ‘+DATADG/csdb/datafile/undotbs1.262.819979889‘;
5> set newname for datafile 4 to ‘+DATADG/csdb/datafile/undotbs2.264.819979905‘;
6> set newname for datafile 5 to ‘+DATADG/csdb/datafile/users.265.819979913‘;
7> restore database;
8> switch datafile all;
9> recover database;
10> }

executing command: SET NEWNAME
released channel: ORA_DISK_1

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

Starting restore at 07-JUL-13
Starting implicit crosscheck backup at 07-JUL-13
allocated channel: ORA_DISK_1
Crosschecked 10 objects
Finished implicit crosscheck backup at 07-JUL-13

Starting implicit crosscheck copy at 07-JUL-13
using channel ORA_DISK_1
Finished implicit crosscheck copy at 07-JUL-13

searching for all files in the recovery area
cataloging files...
no files cataloged

using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00002 to +DATADG/csdb/datafile/sysaux.261.819979871
channel ORA_DISK_1: restoring datafile 00003 to +DATADG/csdb/datafile/undotbs1.262.819979889
channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/CSDB_819991120_6.bk
channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/CSDB_819991120_6.bk tag=ORCL_HOT_DB_BK
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:46
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to +DATADG/csdb/datafile/system.260.819979847
channel ORA_DISK_1: restoring datafile 00004 to +DATADG/csdb/datafile/undotbs2.264.819979905
channel ORA_DISK_1: restoring datafile 00005 to +DATADG/csdb/datafile/users.265.819979913
channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/CSDB_819991120_5.bk
channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/CSDB_819991120_5.bk tag=ORCL_HOT_DB_BK
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:35
Finished restore at 07-JUL-13

datafile 1 switched to datafile copy
input datafile copy RECID=6 STAMP=820112831 file name=+DATADG/csdb/datafile/system.284.820112797
datafile 2 switched to datafile copy
input datafile copy RECID=7 STAMP=820112831 file name=+DATADG/csdb/datafile/sysaux.282.820112751
datafile 3 switched to datafile copy
input datafile copy RECID=8 STAMP=820112831 file name=+DATADG/csdb/datafile/undotbs1.283.820112751
datafile 4 switched to datafile copy
input datafile copy RECID=9 STAMP=820112831 file name=+DATADG/csdb/datafile/undotbs2.285.820112797
datafile 5 switched to datafile copy
input datafile copy RECID=10 STAMP=820112831 file name=+DATADG/csdb/datafile/users.286.820112797

Starting recover at 07-JUL-13
using channel ORA_DISK_1

starting media recovery

channel ORA_DISK_1: starting archived log restore to default destination
channel ORA_DISK_1: restoring archived log
archived log thread=1 sequence=15
channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/arc_819991156_9.bk
channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/arc_819991156_9.bk tag=TAG20130705T151916
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_1_seq_15.256.820112833 thread=1 sequence=15
channel ORA_DISK_1: starting archived log restore to default destination
channel ORA_DISK_1: restoring archived log
archived log thread=2 sequence=2
channel ORA_DISK_1: restoring archived log
archived log thread=1 sequence=16
channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/arc_819991156_10.bk
channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/arc_819991156_10.bk tag=TAG20130705T151916
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_2_seq_2.257.820112835 thread=2 sequence=2
channel default: deleting archived log(s)
archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_1_seq_15.256.820112833 RECID=5 STAMP=820112832
archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_1_seq_16.258.820112835 thread=1 sequence=16
channel default: deleting archived log(s)
archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_2_seq_2.257.820112835 RECID=7 STAMP=820112834
channel ORA_DISK_1: starting archived log restore to default destination
channel ORA_DISK_1: restoring archived log
archived log thread=2 sequence=3
channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/arc_819991158_11.bk
channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/arc_819991158_11.bk tag=TAG20130705T151916
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_2_seq_3.257.820112837 thread=2 sequence=3
channel default: deleting archived log(s)
archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_2_seq_3.257.820112837 RECID=8 STAMP=820112835
unable to find archived log
archived log thread=2 sequence=4
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 07/07/2013 01:07:16
RMAN-06054: media recovery requesting unknown archived log for thread 2 with sequence 4 and starting SCN of 323980

(10) 更改REDO LOG位置信息

SQL> select member from v$logfile;

MEMBER

+DATA/csdb/redo01.log
+DATA/csdb/redo02.log
+DATA/csdb/redo03.log
+DATA/csdb/redo04.log

SQL> alter database rename file ‘+DATA/csdb/redo01.log‘ to ‘+DATADG/csdb/redo01.log‘;

Database altered.

SQL> alter database rename file ‘+DATA/csdb/redo02.log‘ to ‘+DATADG/csdb/redo02.log‘;

Database altered.

SQL> alter database rename file ‘+DATA/csdb/redo03.log‘ to ‘+DATADG/csdb/redo03.log‘;

Database altered.

SQL> alter database rename file ‘+DATA/csdb/redo04.log‘ to ‘+DATADG/csdb/redo04.log‘;

Database altered.

(11) 打開數據庫

SQL> alter database open resetlogs;

Database altered.

(12) 更改TEMP表空間文件位置

SQL> select name from v$tempfile;

NAME

+DATA/csdb/tempfile/temp.263.819979895

SQL> alter tablespace temp add tempfile ‘+DATADG‘;

Tablespace altered.

SQL> alter tablespace temp drop tempfile ‘+DATA/csdb/tempfile/temp.263.819979895‘;

Tablespace altered

4.4 完成恢復操作

(1) 在其他RAC節點上更改OCR路徑

[root@rac2 ~]# vi /etc/oracle/ocr.loc

ocrconfig_loc=+SYSTEMDG
local_only=FALSE

(2) 在恢復節點上重啟CRS

[root@rac1 ~]# crsctl stop crs
[root@rac1 ~]# crsctl start has

(3) 在其他節點上啟動CRS

[root@rac2 ~]# crsctl start crs

ORACLE 11G RAC ASM磁盤組全部丟失後的恢復