• 啟動RAC

手工啟動按照HAS, cluster, database的順序啟動，具體命令如下：

• 啟動HAS(High Availability Services)，必須以root用戶

[root@orcl-1 ~]# crsctl start has
CRS-4124: Oracle High Availability Services startup failed.
CRS-4000: Command Start failed, or completed with errors.

報錯原因是因為11g的一個bug，重新開一個窗口執行/bin/dd if=/var/tmp/.oracle/npohasd f=/dev/null bs=1024 count=1
[root@orcl-1 ~]# crsctl start has
CRS-4640: Oracle High Availability Services is already active
CRS-4000: Command Start failed, or completed with errors.

以上has啟動命令需要在每個節點分別執行

• 啟動集群（cluster）

[root@orcl-1 ~]# crsctl start cluster -all --所有節點同時啟動
CRS-5702: Resource ‘ora.evmd‘ is already running on ‘orcl-1‘
CRS-5702: Resource ‘ora.crsd‘ is already running on ‘orcl-1‘

或者只啟動指定節點的
[root@orcl-1 ~]# ./crsctl start cluster -n orcl1 orcl2 --兩個節點同時啟動

• 啟動數據庫（數據庫名為orcl）：

[root@orcl-1 ~]# srvctl start database -d orcl ---啟動所有節點上的實例
PRCC-1014 : orcl was already running

或者每個節點登錄數據庫後再執行

SQL>startup

[root@orcl-1 ~]# srvctl status asm ---查看ASM狀態
ASM is running on orcl-1,orcl-2

[root@orcl-1 ~]# crs_stat -t -v ---查看集群資源狀態（oc4j、gsd默認服務是禁用的）
Name Type R/RA F/FT Target State Host
----------------------------------------------------------------------
ora.FRA.dg ora....up.type 0/5 0/ ONLINE ONLINE orcl-1
ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE orcl-1
ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE orcl-1
ora.OCR.dg ora....up.type 0/5 0/ ONLINE ONLINE orcl-1
ora.RACDB.dg ora....up.type 0/5 0/ ONLINE ONLINE orcl-1
ora....DISK.dg ora....up.type 0/5 0/ ONLINE ONLINE orcl-1
ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE orcl-1
ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE orcl-1
ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE
ora....network ora....rk.type 0/5 0/ ONLINE ONLINE orcl-1
ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE orcl-1
ora....SM1.asm application 0/5 0/0 ONLINE ONLINE orcl-1
ora....-1.lsnr application 0/5 0/0 ONLINE ONLINE orcl-1
ora.orcl-1.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.orcl-1.ons application 0/3 0/0 ONLINE ONLINE orcl-1
ora.orcl-1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE orcl-1
ora....SM2.asm application 0/5 0/0 ONLINE ONLINE orcl-2
ora....-2.lsnr application 0/5 0/0 ONLINE ONLINE orcl-2
ora.orcl-2.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.orcl-2.ons application 0/3 0/0 ONLINE ONLINE orcl-2
ora.orcl-2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE orcl-2
ora.orcl.db ora....se.type 0/2 0/1 ONLINE ONLINE orcl-1
ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE orcl-1

[root@orcl-1 ~]# srvctl status database -d orcl ---檢查數據庫狀態
Instance orcl1 is running on node orcl-1
Instance orcl2 is running on node orcl-2

[root@orcl-1 ~]# srvctl status nodeapps --- 檢查節點狀態
VIP orcl-1-vip is enabled
VIP orcl-1-vip is running on node: orcl-1
VIP orcl-2-vip is enabled
VIP orcl-2-vip is running on node: orcl-2
Network is enabled
Network is running on node: orcl-1
Network is running on node: orcl-2
GSD is disabled
GSD is not running on node: orcl-1
GSD is not running on node: orcl-2
ONS is enabled
ONS daemon is running on node: orcl-1
ONS daemon is running on node: orcl-2
eONS is enabled
eONS daemon is running on node: orcl-1
eONS daemon is running on node: orcl-2

votedisk檢查(grid用戶)

[root@orcl-1 ~]# crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 786f6bda41ff4fb8bf2b3e17fd7302c9 (/dev/asm-disk-mpathg) [OCR]
2. ONLINE 8b5315974e484f30bff0eb3f2dfaa443 (/dev/asm-disk-mpathh) [OCR]
3. ONLINE 394cd132329b4f39bf57dc8322e359c2 (/dev/asm-disk-mpathi) [OCR]
4. ONLINE 561d06ffcb3e4fe5bf2b896422a8b5cc (/dev/asm-disk-mpathj) [OCR]
Located 4 voting disk(s).

[grid@orcl-2 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2588
Available space (kbytes) : 259532
ID : 2021652580
Device/File Name : +OCR
Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check bypassed due to non-privileged user

關閉RAC

關閉數據庫（oracle用戶）

[oracle@orcl-2 ~]$ srvctl stop database -d orcl ---停止所有節點上的實例
[oracle@orcl-2 ~]$ srvctl status database -d orcl ---查看數據庫實例狀態
Instance orcl1 is not running on node orcl-1
Instance orcl2 is not running on node orcl-2

或者每個節點登錄數據庫後執行SQL>shutdown immediate

停止集群服務，必須以root用戶

[root@orcl-1 ~]# crsctl stop cluster -all
CRS-2673: Attempting to stop ‘ora.crsd‘ on ‘orcl-2‘
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.OCR.dg‘ on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.FRA.dg‘ on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.RACDB.dg‘ on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.LISTENER.lsnr‘ on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.crsd‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.LISTENER.lsnr‘ on ‘orcl-2‘ succeeded
CRS-2673: Attempting to stop ‘ora.orcl-2.vip‘ on ‘orcl-2‘
CRS-2677: Stop of ‘ora.orcl-2.vip‘ on ‘orcl-2‘ succeeded
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.OCR.dg‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.VOTING_DISK.dg‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.FRA.dg‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.RACDB.dg‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.LISTENER.lsnr‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.LISTENER_SCAN1.lsnr‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.LISTENER.lsnr‘ on ‘orcl-1‘ succeeded
CRS-2673: Attempting to stop ‘ora.orcl-1.vip‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.LISTENER_SCAN1.lsnr‘ on ‘orcl-1‘ succeeded
CRS-2673: Attempting to stop ‘ora.scan1.vip‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.orcl-1.vip‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.scan1.vip‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.OCR.dg‘ on ‘orcl-2‘ succeeded
CRS-2677: Stop of ‘ora.FRA.dg‘ on ‘orcl-2‘ succeeded
CRS-2677: Stop of ‘ora.OCR.dg‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.FRA.dg‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.RACDB.dg‘ on ‘orcl-2‘ succeeded
CRS-2673: Attempting to stop ‘ora.asm‘ on ‘orcl-2‘
CRS-2677: Stop of ‘ora.VOTING_DISK.dg‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.asm‘ on ‘orcl-2‘ succeeded
CRS-2673: Attempting to stop ‘ora.ons‘ on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.eons‘ on ‘orcl-2‘
CRS-2677: Stop of ‘ora.RACDB.dg‘ on ‘orcl-1‘ succeeded
CRS-2673: Attempting to stop ‘ora.asm‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.asm‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.ons‘ on ‘orcl-2‘ succeeded
CRS-2673: Attempting to stop ‘ora.net1.network‘ on ‘orcl-2‘
CRS-2677: Stop of ‘ora.net1.network‘ on ‘orcl-2‘ succeeded
CRS-2677: Stop of ‘ora.eons‘ on ‘orcl-2‘ succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on ‘orcl-2‘ has completed
CRS-2673: Attempting to stop ‘ora.ons‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.eons‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.crsd‘ on ‘orcl-2‘ succeeded
CRS-2673: Attempting to stop ‘ora.cssdmonitor‘ on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.ctssd‘ on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.evmd‘ on ‘orcl-2‘
CRS-2673: Attempting to stop ‘ora.asm‘ on ‘orcl-2‘
CRS-2677: Stop of ‘ora.cssdmonitor‘ on ‘orcl-2‘ succeeded
CRS-2677: Stop of ‘ora.ons‘ on ‘orcl-1‘ succeeded
CRS-2673: Attempting to stop ‘ora.net1.network‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.net1.network‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.evmd‘ on ‘orcl-2‘ succeeded
CRS-2677: Stop of ‘ora.eons‘ on ‘orcl-1‘ succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on ‘orcl-1‘ has completed
CRS-2677: Stop of ‘ora.crsd‘ on ‘orcl-1‘ succeeded
CRS-2673: Attempting to stop ‘ora.cssdmonitor‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.ctssd‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.evmd‘ on ‘orcl-1‘
CRS-2673: Attempting to stop ‘ora.asm‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.cssdmonitor‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.evmd‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.asm‘ on ‘orcl-2‘ succeeded
CRS-2677: Stop of ‘ora.ctssd‘ on ‘orcl-2‘ succeeded
CRS-2673: Attempting to stop ‘ora.cssd‘ on ‘orcl-2‘
CRS-2677: Stop of ‘ora.ctssd‘ on ‘orcl-1‘ succeeded
CRS-2677: Stop of ‘ora.cssd‘ on ‘orcl-2‘ succeeded
CRS-2673: Attempting to stop ‘ora.diskmon‘ on ‘orcl-2‘
CRS-2677: Stop of ‘ora.asm‘ on ‘orcl-1‘ succeeded
CRS-2673: Attempting to stop ‘ora.cssd‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.cssd‘ on ‘orcl-1‘ succeeded
CRS-2673: Attempting to stop ‘ora.diskmon‘ on ‘orcl-1‘
CRS-2677: Stop of ‘ora.diskmon‘ on ‘orcl-2‘ succeeded
CRS-2677: Stop of ‘ora.diskmon‘ on ‘orcl-1‘ succeeded

或者每個節點分別執行

[root@orcl-1]# crsctl stop cluster ----停止本節點集群服務，

也可以指定停止節點：

[root@orcl-1]# ./crsctl stop cluster -n orcl1 orcl2

[root@orcl-1 ~]# crsctl config crs
CRS-4622: Oracle High Availability Services autostart is enabled. ---檢查crs配置為是否自動啟動
[root@orcl-1 ~]# crsctl disable crs
CRS-4621: Oracle High Availability Services autostart is disabled. ---禁止CRS系統重啟而自動啟動
[root@orcl-1 ~]# crsctl enable crs
CRS-4622: Oracle High Availability Services autostart is enabled. ---啟用CRS系統重啟而自動啟動（每個節點分別執行）

oracle 11g RAC 的基本操作（一）------啟動與關閉