Oracle 11g RAC  故障之--Instance 啟動失敗

系統環境：

作業系統：RedHat EL5.5

Cluster：           GI 11.2.0.1.0

資料庫軟體：Oracle 11.2.0.1

故障現象：

  新增新的node3的Instance 後，再啟動Instance 時，出現以下故障。

[[email protected] dbs]$ sqlplus '/as sysdba'

sqlplus '/as sysdba'

SQL*Plus: Release 11.2.0.1.0 Production on Thu Apr 17 15:47:41 2014

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

Connected to an idle instance.

[email protected] prod3>startup nomount;

ORA-15081: failed to submit an I/O operation to a disk

告警日誌：

ORACLE_BASE from environment = /u01/app/oracle

Errors in file /u01/app/oracle/diag/rdbms/pmydb/prod3/trace/prod3_rbal_16308.trc:

ORA-15183: ASMLIB initialization error [driver/agent not installed]

WARNING: FAILED to load library: /opt/oracle/extapi/32/asm/orcl/1/libasm.so

Errors in file /u01/app/oracle/diag/rdbms/pmydb/prod3/trace/prod3_rbal_16308.trc:

ORA-15183: ASMLIB initialization error [driver/agent not installed]

SUCCESS: diskgroup DG1 was mounted

USER (ospid: 16204): terminating the instance due to error 15081

NOTE: dependency between database pmydb and diskgroup resource ora.DG1.dg is established

Instance terminated by USER, pid = 16204

檢視相關trace 檔案：

[[email protected] node3]# tail  /u01/app/oracle/diag/rdbms/pmydb/prod3/trace/prod3_rbal_16308.trc

*** 2014-04-17 15:47:55.374

*** SESSION ID:(11.1) 2014-04-17 15:47:55.374

*** CLIENT ID:() 2014-04-17 15:47:55.374

*** SERVICE NAME:() 2014-04-17 15:47:55.374

*** MODULE NAME:() 2014-04-17 15:47:55.374

*** ACTION NAME:() 2014-04-17 15:47:55.374

ERROR: asm_version error. err: driver/agent not installed rc:2

ORA-15183: ASMLIB initialization error [driver/agent not installed]

ORA-15183: ASMLIB initialization error [driver/agent not installed]

已經安裝ASM相關的driver：

[[email protected] ~]# rpm -qa |grep asm

oracleasmlib-2.0.4-1.el5

oracleasm-support-2.1.3-1.el5

oracleasm-2.6.18-194.el5-2.0.5-1.el5

[[email protected] ~]# uname -a

Linux node1 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:43 EDT 2010 i686 i686 i386 GNU/Linux

檢視Oracle 可執行檔案：

[[email protected] asm]# ls -l /u01/app/oracle/product/11.2.0/db_1/bin/oracle

-r-sr-s--x 1 oracle oinstall 173515905 Apr 17 15:07 /u01/app/oracle/product/11.2.0/db_1/bin/oracle

    發現檔案的所屬的group錯誤，Oracle 後臺程序啟動時會執行此檔案，而如果屬於oinstall組，則無權訪問ASM磁碟驅動，從而無法訪問ASM disk。

正常屬性應該是：

[[email protected] ~]$ ls -l /u01/app/oracle/product/11.2.0/db_1/bin/oracle

-r-sr-s--x 1 oracle asmadmin 173515905 Dec 28 18:17 /u01/app/oracle/product/11.2.0/db_1/bin/oracle

修改屬性：

[[email protected] ~]# chown oracle:asmadmin /u01/app/oracle/product/11.2.0/db_1/bin/oracle

[[email protected] asm]# ls -l /u01/app/oracle/product/11.2.0/db_1/bin/oracle

-r-xr-x--x 1 oracle asmadmin 173515905 Apr 17 15:07 /u01/app/oracle/product/11.2.0/db_1/bin/oracle

[[email protected] asm]# chmod 6551  /u01/app/oracle/product/11.2.0/db_1/bin/oracle

[[email protected] asm]# ls -l /u01/app/oracle/product/11.2.0/db_1/bin/oracle

-r-sr-s--x 1 oracle asmadmin 173515905 Apr 17 15:07 /u01/app/oracle/product/11.2.0/db_1/bin/oracle

@更改完成後，再啟動Instance ，Instance 啟動正常！