SRVCTL Fails to Start Instance with ORA-17503 ORA-27303 But sqlplus Startup is Fine [ID 1322959.1]

修改時間 23-JAN-2012     型別 PROBLEM     狀態 PUBLISHED

Applies to:

Oracle Server - Enterprise Edition - Version: 11.2.0.1 and later   [Release: 11.2 and later ]
Information in this document applies to any platform.

Symptoms

RAC Instance can be started using sqlplus but fails when using srvctl:

Case I:

$ srvctl start database -d cad
PRCR-1079 : Failed to start resource ora.cad.db
CRS-5017: The resource action "ora.cad.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+RAC/cad/spfilecad.ora'
ORA-17503: ksfdopn:2 Failed to open file +RAC/cad/spfilecad.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 1002 (oinstall), current egid = 1001 (dba)

CRS-2674: Start of 'ora.cad.db' on 'racnode2' failed
CRS-2632: There are no more servers to try to place resource 'ora.cad.db' on that would satisfy its placement policy

Case II

$ srvctl start database -d nar
PRCR-1079 : Failed to start resource ora.nar.db
CRS-5017: The resource action "ora.nar.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+NAR_DG/nar/spfilenar.ora'
ORA-17503: ksfdopn:2 Failed to open file +NAR_DG/nar/spfilenar.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 59800 (oinstall), current egid = 59802 (asmadmin)
. For details refer to "(:CLSN00107:)" in "/oracle/GRID/11203/log/dog1104/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-2674: Start of 'ora.nar.db' on 'racnode1' failed

Cause

For Case I, the issue is caused wrong group (dba) is chosen for "ASM Database Administrator","ASM Instance Administration Operator" and "ASM Instance Administrator"
group during installing grid infrastructure, while the ASM disks are owned by oinstall group.

This same issue is documented in
Bug 9786198 - SRVCTL START DATABASE ORA-0178 FAILURE IN PROCESSING SYSTEM PARAMETERS

For Case II, the issue is caused by setuid and setgid bit missing for oracle binary under GRID_HOME, eg:

$ ls -l $GRID_HOME/bin/oracle
-rwxr-x--x oracle oinstall    166648778 Dec  7 15:26           oracle The missing setuid and setgid bit causing oracle user access failure to ASM diskgroup even though the ASMADMIN group is set correctly.

Solution

For case I,

Either reinstall Grid Infrastructure, select correct group for ASM Instance Administrator - ASMADMIN (oinstall in this case which is the ASM disk owning group)
or
stop srvctl modify oracle binary group from oinstall to dba by renaming $GRID_HOME/bin/setasmgid to $GRID_HOME/bin/setasmgid.orig. Restore the group ownership of oracle binary to oinstall and startup instance using srvctl.

For case II, 

Add setuid and setgid bit for oracle binary under $GRID_HOME/bin:
$ chmod 6755 $GRID_HOME/bin/oracle

The oracle binary for both GRID_HOME/bin and RDBMS ORACLE_HOME/bin should have 6755 permission, eg: -rwsr-s--x