1. 程式人生 > >oracle12C RAC GI + UDEV + ASM 在centos6下安裝詳細步驟

oracle12C RAC GI + UDEV + ASM 在centos6下安裝詳細步驟

Centos配置網路yum源:

Centos配置安裝vnc客戶端和服務端:

Centos配置dns伺服器:

Centos公網雙網絡卡繫結bond0

Centos下安裝GI前準備詳細執行步驟:

Centos下安裝oracle12c需要的依賴包詳細安裝過程:

Centos下安裝oracle12c節點1完成後檢查過程:

Centos下安裝oracle12c四個節點都完成共享磁碟連線後對共享磁碟分割槽詳細過程:

centos6.4 /etc/resolv.conf檔案改了重啟網路就自動還原了

http://blog.csdn.net/kadwf123/article/details/78786947

作業系統Centos6.4:

安裝前網路規劃:

1、使用之前已經配置好的公網bond0的ip通過crt登陸主機

2、編輯eth2的配置檔案

[[email protected] network-scripts]# vi ifcfg-eth2

DEVICE=eth2 HWADDR=08:00:27:18:29:48 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=none IPV6INIT=no USERCTL=no IPADDR=10.0.10.1 NETMASK=255.255.255.0 GATEWAY=192.168.0.1

3、編輯eth3的配置檔案

[plain] view plain copy print?

  1. <code class="language-plain">[[email protected] network-scripts]# vi ifcfg-eth3  
  2. DEVICE=eth3  
  3. HWADDR=08:00:27:59:1e:79  
  4. TYPE=Ethernet  
  5. ONBOOT=yes  
  6. BOOTPROTO=none  
  7. IPADDR=10.0.10.2  
  8. NETMASK=255.255.255.0  
  9. GATEWAY=192.168.0.1  
  10. IPV6INIT=no  
  11. USERCTL=no</code>  
  1. [[email protected] network-scripts]# vi ifcfg-eth3

  2. DEVICE=eth3

  3. HWADDR=08:00:27:59:1e:79

  4. TYPE=Ethernet

  5. ONBOOT=yes

  6. BOOTPROTO=none

  7. IPADDR=10.0.10.2

  8. NETMASK=255.255.255.0

  9. GATEWAY=192.168.0.1

  10. IPV6INIT=no

  11. USERCTL=no


4、bootproto=no表示手動設定ip地址。onboot=yes表示開機自動載入網絡卡。

注意設定ipaddr、geteway、netmask選項。

5、設定完成重啟網路服務。

  1. [[email protected] network-scripts]# service network restart

  2. 正在關閉介面 bond0: [確定]

  3. 正在關閉介面 eth2: [確定]

  4. 正在關閉介面 eth3: [確定]

  5. 關閉環回介面: [確定]

  6. 彈出環回介面: [確定]

  7. 彈出介面 bond0: [確定]

  8. 彈出介面 eth2: [確定]

  9. 彈出介面 eth3: [確定]


6、ifconfig檢視eth2和eth3網絡卡IP地址是否正確;

  1. [[email protected] network-scripts]# ifconfig

  2. bond0 Link encap:Ethernet HWaddr 08:00:27:FC:7E:5B

  3. inet addr:192.168.0.51 Bcast:192.168.0.255 Mask:255.255.255.0

  4. inet6 addr: fe80::a00:27ff:fefc:7e5b/64 Scope:Link

  5. UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1

  6. RX packets:5185 errors:0 dropped:0 overruns:0 frame:0

  7. TX packets:2571 errors:0 dropped:0 overruns:0 carrier:0

  8. collisions:0 txqueuelen:0

  9. RX bytes:463220 (452.3 KiB) TX bytes:319388 (311.9 KiB)

  10. eth0 Link encap:Ethernet HWaddr 08:00:27:FC:7E:5B

  11. UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1

  12. RX packets:4262 errors:0 dropped:0 overruns:0 frame:0

  13. TX packets:1912 errors:0 dropped:0 overruns:0 carrier:0

  14. collisions:0 txqueuelen:1000

  15. RX bytes:377145 (368.3 KiB) TX bytes:247430 (241.6 KiB)

  16. eth1 Link encap:Ethernet HWaddr 08:00:27:FC:7E:5B

  17. UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1

  18. RX packets:923 errors:0 dropped:0 overruns:0 frame:0

  19. TX packets:659 errors:0 dropped:0 overruns:0 carrier:0

  20. collisions:0 txqueuelen:1000

  21. RX bytes:86075 (84.0 KiB) TX bytes:71958 (70.2 KiB)

  22. eth2 Link encap:Ethernet HWaddr 08:00:27:18:29:48

  23. inet addr:10.0.10.1 Bcast:10.0.10.255 Mask:255.255.255.0

  24. inet6 addr: fe80::a00:27ff:fe18:2948/64 Scope:Link

  25. UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

  26. RX packets:1036 errors:0 dropped:0 overruns:0 frame:0

  27. TX packets:123 errors:0 dropped:0 overruns:0 carrier:0

  28. collisions:0 txqueuelen:1000

  29. RX bytes:92021 (89.8 KiB) TX bytes:13614 (13.2 KiB)

  30. eth3 Link encap:Ethernet HWaddr 08:00:27:59:1E:79

  31. inet addr:10.0.10.2 Bcast:10.0.10.255 Mask:255.255.255.0

  32. inet6 addr: fe80::a00:27ff:fe59:1e79/64 Scope:Link

  33. UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

  34. RX packets:937 errors:0 dropped:0 overruns:0 frame:0

  35. TX packets:50 errors:0 dropped:0 overruns:0 carrier:0

  36. collisions:0 txqueuelen:1000

  37. RX bytes:87752 (85.6 KiB) TX bytes:3208 (3.1 KiB)

  38. lo Link encap:Local Loopback

  39. inet addr:127.0.0.1 Mask:255.0.0.0

  40. inet6 addr: ::1/128 Scope:Host

  41. UP LOOPBACK RUNNING MTU:16436 Metric:1

  42. RX packets:8 errors:0 dropped:0 overruns:0 frame:0

  43. TX packets:8 errors:0 dropped:0 overruns:0 carrier:0

  44. collisions:0 txqueuelen:0

  45. RX bytes:728 (728.0 b) TX bytes:728 (728.0 b)

按照上面配置的私網ip,因為ip在一個網段,所以會檢查出錯誤

詳細錯誤說明如下:

  1. Node Connectivity - This is a prerequisite condition to test whether connectivity exists amongst all the nodes. The connectivity is being tested for the subnets "10.0.10.0,10.0.10.0,192.168.0.0"

  2. Check Failed on Nodes: [rac2, ?rac1, ?rac4, ?rac3]

  3. Verification result of failed node: rac2 ?Details:

  4. ?-?

  5. PRVG-11073 : Subnet on interface "eth2" of node "rac1" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac2" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac3" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac4" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. ?- Cause:?Cause Of Problem Not Available ?- Action:?User Action Not Available

  6. Back to Top

  7. Verification result of failed node: rac1 ?Details:

  8. ?-?

  9. PRVG-11073 : Subnet on interface "eth2" of node "rac1" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac2" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac3" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac4" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. ?- Cause:?Cause Of Problem Not Available ?- Action:?User Action Not Available

  10. Back to Top

  11. Verification result of failed node: rac4 ?Details:

  12. ?-?

  13. PRVG-11073 : Subnet on interface "eth2" of node "rac1" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac2" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac3" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac4" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. ?- Cause:?Cause Of Problem Not Available ?- Action:?User Action Not Available

  14. Back to Top

  15. Verification result of failed node: rac3 ?Details:

  16. ?-?

  17. PRVG-11073 : Subnet on interface "eth2" of node "rac1" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac2" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac3" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. PRVG-11073 : Subnet on interface "eth2" of node "rac4" is overlapping with the subnet on interface "eth3". IP address range ["10.0.10.0"-"10.0.10.255"] is overlapping with IP address range ["10.0.10.0"-"10.0.10.255"]. ?- Cause:?Cause Of Problem Not Available ?- Action:?User Action Not Available

  18. Back to Top

如果遇到上面的錯誤,你可以選擇忽略。如果不想忽略,也可以選擇把兩塊私網ip配置成不同網段的ip地址。比如eth2配置成10.0.10.1,eth3配置成10.0.11.2就可以了。

7、本地ping

  1. [[email protected] network-scripts]# ping 10.0.10.1

  2. PING 10.0.10.1 (10.0.10.1) 56(84) bytes of data.

  3. 64 bytes from 10.0.10.1: icmp_seq=1 ttl=64 time=0.024 ms

  4. 64 bytes from 10.0.10.1: icmp_seq=2 ttl=64 time=0.030 ms

  5. ^C

  6. --- 10.0.10.1 ping statistics ---

  7. 2 packets transmitted, 2 received, 0% packet loss, time 1893ms

  8. rtt min/avg/max/mdev = 0.024/0.027/0.030/0.003 ms

  9. [[email protected] network-scripts]# ping 10.0.10.2

  10. PING 10.0.10.2 (10.0.10.2) 56(84) bytes of data.

  11. 64 bytes from 10.0.10.2: icmp_seq=1 ttl=64 time=0.023 ms

  12. 64 bytes from 10.0.10.2: icmp_seq=2 ttl=64 time=0.032 ms

  13. 64 bytes from 10.0.10.2: icmp_seq=3 ttl=64 time=0.038 ms

  14. ^C

  15. --- 10.0.10.2 ping statistics ---

  16. 3 packets transmitted, 3 received, 0% packet loss, time 2007ms

  17. rtt min/avg/max/mdev = 0.023/0.031/0.038/0.006 ms

  18. [[email protected] network-scripts]#

8、ok,私網配置沒問題。

9、確認該節點主機名是rac1

vi /etc/sysconfig/network

NETWORKDING=yes


HOSTNAME=rac1

NOZEROCONF=yes

命令:

hostname rac1

更改主機名立即生效

10、配置節點rac1使用dns伺服器進行解析

vi /etc/resolv.conf

options attempts: 2

options timeout: 1

search taryartar.com

nameserver 192.168.0.88

nameserver 192.168.0.1

nameserver 8.8.8.8

11、驗證dns是否能夠生效

正向解析:

host rac1

nslookup rac1

反向解析:

nslookup 192.168.0.51

都沒問題。

  1. [[email protected] network-scripts]# nslookup rac1

  2. Server: 192.168.0.88

  3. Address: 192.168.0.88#53

  4. Name: rac1.taryartar.com

  5. Address: 192.168.0.51

  6. [[email protected] network-scripts]# nslookup 192.168.0.51

  7. Server: 192.168.0.88

  8. Address: 192.168.0.88#53

  9. 51.0.168.192.in-addr.arpa name = rac1.taryartar.com.

  10. [[email protected] network-scripts]# host rac1

  11. rac1.taryartar.com has address 192.168.0.51

  12. [[email protected] network-scripts]#

12、建立使用者和使用者組

  1. [[email protected] network-scripts]# groupadd -g 1000 oinstall

  2. [[email protected] network-scripts]# groupadd -g 1031 dba

  3. [[email protected] network-scripts]# groupadd -g 1032 asmdba

  4. [[email protected] network-scripts]# useradd -u 1101 -g oinstall -G dba,asmdba oracle

  5. [[email protected] network-scripts]# useradd -u 1100 -g oinstall -G asmdba grid

  6. [[email protected] network-scripts]# mkdir -p /taryartar/12c/grid_base

  7. [[email protected] network-scripts]# mkdir -p /taryartar/12c/grid_home

  8. [[email protected] network-scripts]# mkdir -p /taryartar/12c/db_base/db_home

  9. [[email protected] network-scripts]# chown -R grid:oinstall /taryartar/12c/grid_base

  10. [[email protected] network-scripts]# chown -R grid:oinstall /taryartar/12c/grid_home

  11. [[email protected] network-scripts]# chown -R oracle:oinstall /taryartar/12c/db_base

  12. [[email protected] network-scripts]# chmod -R 775 /taryartar/12c/db_base

  13. [[email protected] network-scripts]# chmod -R 775 /taryartar/12c/grid_base

  14. [[email protected] network-scripts]# chmod -R 775 /taryartar/12c/grid_home

  15. [[email protected] network-scripts]#

13、系統配置

Uname -a

Lsb_release

Each system must meet the following minimum memory requirements:

· 

At least 4 GB of RAM for Oracle Grid Infrastructure for a Cluster installations, including installations where you plan to install Oracle RAC.

· 

· 

Swap space equivalent to the multiple of the available RAM, as indicated in the following table:

· 

Table 2-1 Swap Space Required for 64-bit Linux and Linux on System z

· 

Available RAM

Swap Space Required

Between 4 GB and 16 GB

Equal to RAM

More than 16 GB

16 GB of RAM

14、交換分割槽不夠,可以如下擴充套件

  1. [[email protected] selinux]# dd if=/dev/zero of=/root/swapfile1 bs=1M count=1024

  2. 記錄了500+0 的讀入

  3. 記錄了500+0 的寫出

  4. 524288000位元組(524 MB)已複製,1.34808 秒,389 MB/秒

  5. [[email protected] selinux]# chmod 600 /root/swapfile1

  6. [[email protected] selinux]# mkswap /root/swapfile1

  7. Setting up swapspace version 1, size = 511996 KiB

  8. no label, UUID=a44009c0-9d5e-4ea5-b9db-f8ae81b16a0b

  9. [[email protected] selinux]# swapon /root/swapfile1

  1. [[email protected] selinux]# free -m

  2. total used free shared buffers cached

  3. Mem: 1500 1428 71 0 20 1238

  4. -/+ buffers/cache: 169 1331

  5. Swap: 4007 0 4007

要想下次重啟仍然生效,需要寫入下面的配置檔案

vi /etc/fstab

新增下面一行

/root/swapfile1 swap swap defaults 0 0


15、注意叢集安裝可能受防火牆和selinux干擾,最好關閉防火牆和selinux,否則出一些某明奇妙的問題排查起來很麻煩

關閉防火牆並關閉自啟動

  1. [[email protected] yum.repos.d]# service iptables stop

  2. iptables:清除防火牆規則:[確定]

  3. iptables:將鏈設定為政策 ACCEPT:filter [確定]

  4. iptables:正在解除安裝模組:[確定]

  5. [[email protected] yum.repos.d]# chkconfig iptables off


16、關閉selinux

  1. [[email protected] yum.repos.d]# getenforce

  2. Enforcing

  3. [[email protected] yum.repos.d]# setenforce 0

  4. [[email protected] yum.repos.d]# getenforce

  5. Permissive

vi /etc/selinux/config

修改SELINUX的值為permissive

SELINUX=permissive

17、關閉ipv6

  1. [[email protected] selinux]# service ip6tables stop

  2. ip6tables:清除防火牆規則:[確定]

  3. ip6tables:將 chains 設定為 ACCEPT 策略:filter [確定]

  4. :正在解除安裝模組:[確定]

  5. [[email protected] selinux]# chkconfig ip6tables off

  6. [[email protected] selinux]#

18、修改ssh連線配置檔案

vi /etc/ssh/sshd_config

新增

LoginGraceTime 0

表示遠端連線節點不會被關閉連線。


19、安裝rpm包

針對redhat6(同centos6)官網指定安裝包如下:

Packages for Oracle Linux 6 and Red Hat Enterprise Linux 6

The following packages (or later versions) must be installed:

binutils-2.20.51.0.2-5.11.el6 (x86_64)

compat-libcap1-1.10-1 (x86_64)

compat-libstdc++-33-3.2.3-69.el6 (x86_64)

compat-libstdc++-33-3.2.3-69.el6.i686

gcc-4.4.4-13.el6 (x86_64)

gcc-c++-4.4.4-13.el6 (x86_64)

glibc-2.12-1.7.el6 (i686)

glibc-2.12-1.7.el6 (x86_64)

glibc-devel-2.12-1.7.el6 (x86_64)

glibc-devel-2.12-1.7.el6.i686

ksh

libgcc-4.4.4-13.el6 (i686)

libgcc-4.4.4-13.el6 (x86_64)

libstdc++-4.4.4-13.el6 (x86_64)

libstdc++-4.4.4-13.el6.i686

libstdc++-devel-4.4.4-13.el6 (x86_64)

libstdc++-devel-4.4.4-13.el6.i686

libaio-0.3.107-10.el6 (x86_64)

libaio-0.3.107-10.el6.i686

libaio-devel-0.3.107-10.el6 (x86_64)

libaio-devel-0.3.107-10.el6.i686

libXext-1.1 (x86_64)

libXext-1.1 (i686)

libXtst-1.0.99.2 (x86_64)

libXtst-1.0.99.2 (i686)

libX11-1.3 (x86_64)

libX11-1.3 (i686)

libXau-1.0.5 (x86_64)

libXau-1.0.5 (i686)

libxcb-1.5 (x86_64)

libxcb-1.5 (i686)

libXi-1.3 (x86_64)

libXi-1.3 (i686)

make-3.81-19.el6

sysstat-9.0.4-11.el6 (x86_64)

nfs-utils-1.2.3-15.0.1


老老實實配置好yum源然後一個個安裝,連結如下:

配置yum源:

安裝依賴包:

20、修改環境變數

為grid使用者設定環境變數
Su - grid
Echo $SHELL
Vi /home/grid/.bash_profile

新增下列內容:

export ORACLE_SID=+ASM1
export ORACLE_BASE=/taryartar/12c/grid_base
export ORACLE_HOME=/taryartar/12c/grid_home
export GRID_HOME=$ORACLE_HOME
export ORACLE_TERM=xterm
export TMP=/tmp
export TMPDIR=$TMP
#PATH=$PATH:$HOME/bin
export PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export ORACLE_HOSTNAME=rac1.taryartar.com
export DB_UNIQUE_NAME=tar
export CVUQDISK_GRP=oinstall
umask 022

[[email protected] ~]$ vi .bash_profile

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH

#add by wufan 2017/10/15
export ORACLE_SID=+ASM1
export ORACLE_BASE=/taryartar/12c/grid_base
export ORACLE_HOME=/taryartar/12c/grid_home
export GRID_HOME=$ORACLE_HOME
export ORACLE_TERM=xterm
export TMP=/tmp
export TMPDIR=$TMP
#PATH=$PATH:$HOME/bin
export PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export ORACLE_HOSTNAME=rac1.taryartar.com
export DB_UNIQUE_NAME=tar
export CVUQDISK_GRP=oinstall
umask 022

使環境變數立即生效:

source .bash_profile

21、修改oracle環境變數

Su - oracle
Echo $SHELL
Vi /home/oracle/.bash_profile

新增下列內容:

export ORACLE_BASE=/taryartar/12c/db_base
export ORACLE_HOME=$ORACLE_BASE/db_home
export ORACLE_SID=tar1
export ORACLE_OWNER=oracle
export ORACLE_TERM=vt100
export PATH=$PATH:$ORACLE_HOME/bin
export PATH=$ORACLE_HOME/Apache/Apache/bin:$PATH
export BASE_PATH=/usr/sbin:$PATH:$BASE_PATH
export PATH=$BASE_PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib
export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib
export CLASSPATH
export ORACLE_HOSTNAME=rac1.taryartar.com
export DB_UNIQUE_NAME=tar
export CVUQDISK=oinstall
umask 022

  1. [[email protected] ~]$ vi .bash_profile

  2. # .bash_profile

  3. # Get the aliases and functions

  4. if [ -f ~/.bashrc ]; then

  5. . ~/.bashrc

  6. fi

  7. # User specific environment and startup programs

  8. PATH=$PATH:$HOME/bin

  9. export PATH

  10. #add by wufan 2017/10/15

  11. export ORACLE_BASE=/taryartar/12c/db_base

  12. export ORACLE_HOME=$ORACLE_BASE/db_home

  13. export ORACLE_SID=tar1

  14. export ORACLE_OWNER=oracle

  15. export ORACLE_TERM=vt100

  16. export PATH=$PATH:$ORACLE_HOME/bin

  17. export PATH=$ORACLE_HOME/Apache/Apache/bin:$PATH

  18. export BASE_PATH=/usr/sbin:$PATH:$BASE_PATH

  19. export PATH=$BASE_PATH

  20. LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib

  21. export LD_LIBRARY_PATH

  22. CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

  23. CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib

  24. export CLASSPATH

  25. export ORACLE_HOSTNAME=rac1.taryartar.com

  26. export DB_UNIQUE_NAME=tar

  27. export CVUQDISK=oinstall

  28. umask 022

使環境變數立即生效:

source .bash_profile

22、修改核心引數

Su - root
Vi /etc/sysctl.conf
先把kernel.shmmax和kernel.shmall
兩個註釋掉
新增下面一段:

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
#實體記憶體的50-60%
kernel.shmmax = 1046898278
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586

[[email protected] ssh]# vi /etc/sysctl.conf
  1. # Controls the maximum shared segment size, in bytes

  2. #kernel.shmmax = 68719476736

  3. # Controls the maximum number of shared memory segments, in pages

  4. #kernel.shmall = 4294967296

  5. ##add by wufan 2017/10/15

  6. fs.aio-max-nr = 1048576

  7. fs.file-max = 6815744

  8. kernel.shmall = 2097152

  9. #實體記憶體的50-60%

  10. kernel.shmmax = 1046898278

  11. kernel.shmmni = 4096

  12. kernel.sem = 250 32000 100 128

  13. net.ipv4.ip_local_port_range = 9000 65500

  14. net.core.rmem_default = 262144

  15. net.core.rmem_max = 4194304

  16. net.core.wmem_default = 262144

  17. net.core.wmem_max = 1048586

使核心引數立即生效

  1. [[email protected] ssh]# sysctl -p

  2. net.ipv4.ip_forward = 0

  3. net.ipv4.conf.default.rp_filter = 1

  4. net.ipv4.conf.default.accept_source_route = 0

  5. kernel.sysrq = 0

  6. kernel.core_uses_pid = 1

  7. net.ipv4.tcp_syncookies = 1

  8. error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key

  9. error: "net.bridge.bridge-nf-call-iptables" is an unknown key

  10. error: "net.bridge.bridge-nf-call-arptables" is an unknown key

  11. kernel.msgmnb = 65536

  12. kernel.msgmax = 65536

  13. fs.aio-max-nr = 1048576

  14. fs.file-max = 6815744

  15. kernel.shmall = 2097152

  16. kernel.shmmax = 1046898278

  17. kernel.shmmni = 4096

  18. kernel.sem = 250 32000 100 128

  19. net.ipv4.ip_local_port_range = 9000 65500

  20. net.core.rmem_default = 262144

  21. net.core.rmem_max = 4194304

  22. net.core.wmem_default = 262144

  23. net.core.wmem_max = 1048586

檢視核心是否生效

  1. [[email protected] ssh]# sysctl -a |grep shmmax

  2. kernel.shmmax = 1046898278


23、修改使用者資源限制

[[email protected] ssh]# vi /etc/security/limits.conf 

新增如下內容:

  1. oracle soft nproc 2047

  2. oracle hard nproc 16384

  3. oracle soft nofile 1024

  4. oracle hard nofile 65536

  5. grid soft nproc 16384

  6. grid hard nproc 16384

  7. grid soft nofile 65536

  8. grid hard nofile 65536

  9. grid soft stack 10240

  10. grid hard stack 10240

nproc 最大程序數限制

nofile 最大檔案數限制


24、至此,節點1安裝前準備就算基本完成,下面需要對節點1重啟後檢查一遍:

http://blog.csdn.net/kadwf123/article/details/78241445

25、對第一個節點執行匯出(需要注意該節點處於停止狀態)。

匯出後對匯出的檔案進行匯入,然後啟動新節點,用crt直接通過節點1的ip進行登入。

26、然後修改主機名和ip

  1. [[email protected] ~]$ vi /etc/sysconfig/network

  2. NETWORKING=yes

  3. HOSTNAME=rac2

[[email protected] ~]# hostname rac2
  1. [[email protected] ~]# hostname

  2. rac2

hostname改成rac2

  1. [[email protected] ~]# vi /etc/sysconfig/network-scripts/ifcfg-bond0

  2. DEVICE=bond0

  3. IPADDR=192.168.0.52

  4. NETMASK=255.255.255.0

  5. USERCTL=no

  6. BOOTPROTO=none

  7. ONBOOT=yes

  8. GATEWAY=192.168.0.1

  9. IPV6INIT=no

  10. TYPE=Ethernet

  11. #DNS1=192.168.0.1

公網ipaddr改成192.168.0.52,其它保持不變

改第一塊私網網絡卡eth2

  1. [[email protected] ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth2

  2. DEVICE=eth2

  3. HWADDR=08:00:27:18:29:48

  4. TYPE=Ethernet

  5. ONBOOT=yes

  6. NM_CONTROLLED=yes

  7. BOOTPROTO=none

  8. IPV6INIT=no

  9. USERCTL=no

  10. IPADDR=10.0.10.3

  11. NETMASK=255.255.255.0

  12. GATEWAY=192.168.0.1

ipaddr改成10.0.10.3,其它保持不變。

改第二塊私網網絡卡eth3

  1. [[email protected] ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth3

  2. DEVICE=eth3

  3. HWADDR=08:00:27:59:1e:79

  4. TYPE=Ethernet

  5. ONBOOT=yes

  6. BOOTPROTO=none

  7. IPADDR=10.0.10.4

  8. NETMASK=255.255.255.0

  9. GATEWAY=192.168.0.1

  10. IPV6INIT=no

  11. USERCTL=no


ipaddr改成10.0.10.4,其它不變。

27、重啟網路服務

[[email protected] ~]# service network restart

重啟後需要重新登陸crt建立到新節點ip192.168.0.52的網路連線。

28、登陸後檢查rac2的dns是否可用。

  1. [[email protected] ~]# nslookup rac3

  2. Server: 192.168.0.88

  3. Address: 192.168.0.88#53

  4. Name: rac3.taryartar.com

  5. Address: 192.168.0.53

  6. [[email protected] ~]# nslookup rac2.taryartar.com

  7. Server: 192.168.0.88

  8. Address: 192.168.0.88#53

  9. Name: rac2.taryartar.com

  10. Address: 192.168.0.52

  11. [[email protected] ~]# nslookup 192.168.0.54

  12. Server: 192.168.0.88

  13. Address: 192.168.0.88#53

  14. 54.0.168.192.in-addr.arpa name = rac4.taryartar.com.

  15. [[email protected] ~]#


ok,正向反向解析都沒問題。

29、修改grid使用者環境變數配置

  1. [[email protected] ~]# su - grid

  2. [[email protected] ~]$ vi .bash_profile

  3. # User specific environment and startup programs

  4. PATH=$PATH:$HOME/bin

  5. export PATH

  6. #add by wufan 2017/1015

  7. export ORACLE_SID=+ASM2

  8. export ORACLE_BASE=/taryartar/12c/grid_base

  9. export ORACLE_HOME=/taryartar/12c/grid_home

  10. export GRID_HOME=$ORACLE_HOME

  11. export ORACLE_TERM=xterm

  12. export TMP=/tmp

  13. export TMPDIR=$TMP

  14. #PATH=$PATH:$HOME/bin

  15. export PATH=/usr/sbin:$PATH:$ORACLE_HOME/bin

  16. export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

  17. export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

  18. export ORACLE_HOSTNAME=rac2.taryartar.com

  19. export DB_UNIQUE_NAME=tar

  20. export CVUQDISK_GRP=oinstall

  21. umask 022

只需修改ORACLE_SID和ORACLE_HOSTNAME兩個變數值

改完立即生效,執行命令

source .bash_profile

檢查更改是否生效:

  1. [[email protected] ~]$ env |grep -i oracle_sid

  2. ORACLE_SID=+ASM2

  3. [[email protected] ~]$ env|grep -i oracle_hostname

  4. ORACLE_HOSTNAME=rac2.taryartar.com

  5. [[email protected] ~]$


30、修改oracle使用者環境變數

  1. [[email protected] ~]# su - oracle

  2. [[email protected] ~]$ vi .bash_profile

  3. PATH=$PATH:$HOME/bin

  4. export PATH

  5. #add by wufan 2017/10/15

  6. export ORACLE_BASE=/taryartar/12c/db_base

  7. export ORACLE_HOME=$ORACLE_BASE/db_home

  8. export ORACLE_SID=tar2

  9. export ORACLE_OWNER=oracle

  10. export ORACLE_TERM=vt100

  11. export PATH=$PATH:$ORACLE_HOME/bin

  12. export PATH=$ORACLE_HOME/Apache/Apache/bin:$PATH

  13. export BASE_PATH=/usr/sbin:$PATH:$BASE_PATH

  14. export PATH=$BASE_PATH

  15. LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib

  16. export LD_LIBRARY_PATH

  17. CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

  18. CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib

  19. export CLASSPATH

  20. export ORACLE_HOSTNAME=rac2.taryartar.com

  21. export DB_UNIQUE_NAME=tar

  22. export CVUQDISK=oinstall

只需修改ORACLE_SID和ORACLE_HOSTNAME兩個變數

改完立即生效,執行命令

source .bash_profile

檢查更改是否生效:

  1. [[email protected] ~]$ env|grep -i oracle_sid

  2. ORACLE_SID=tar2

  3. [[email protected] ~]$ env|grep -i oracle_hostname

  4. ORACLE_HOSTNAME=rac2.taryartar.com

  5. [[email protected] ~]$

31、至此,第二個節點更改完成。

32、按照上述方法建立第3-4個節點。

33、下面配置共享儲存

一共七塊盤,兩塊10g,外加5塊各2g。

34、建立磁碟virtualbox命令建立:

建立第一塊磁碟:

E:\Program Files\Oracle\VirtualBox\VBoxManage createhd --filename E:\實驗環境\12CRAC\V_SHARES\taryartar1.vdi --size 12288 --format VDI --variant Fixed

如果你跟我一樣,virtualbox安裝目錄有空格Program Files,那麼可能遇到命令不存在的問題。為了方便,我直接把virtual安裝目錄直接配置到環境變數PATH中。

這樣就可以直接執行命令了。

好了成功了。建立剩下的六塊盤,附上指令碼。

  1. VBoxManage createhd --filename E:\實驗環境\12CRAC\V_SHARES\taryartar1.vdi --size 12288 --format VDI --variant Fixed

  2. VBoxManage createhd --filename E:\實驗環境\12CRAC\V_SHARES\taryartar2.vdi --size 12288 --format VDI --variant Fixed

  3. VBoxManage createhd --filename E:\實驗環境\12CRAC\V_SHARES\taryartar3.vdi --size 2048 --format VDI --variant Fixed

  4. VBoxManage createhd --filename E:\實驗環境\12CRAC\V_SHARES\taryartar4.vdi --size 2048 --format VDI --variant Fixed

  5. VBoxManage createhd --filename E:\實驗環境\12CRAC\V_SHARES\taryartar5.vdi --size 2048 --format VDI --variant Fixed

  6. VBoxManage createhd --filename E:\實驗環境\12CRAC\V_SHARES\taryartar6.vdi --size 2048 --format VDI --variant Fixed

  7. VBoxManage createhd --filename E:\實驗環境\12CRAC\V_SHARES\taryartar7.vdi --size 2048 --format VDI --variant Fixed

命令說明:

VBoxManage 是virtualbox安裝目錄中的命令,VBoxManage.exe可執行檔案。

--filename指定需要建立的磁碟的檔案位置和名字。

--size指定建立的磁碟大小,單位M。

--format指定磁碟的格式。

35、建立磁碟完成後,需要連線到虛擬機器。

在連線到虛擬機器前,需要先關閉虛擬機器。

連線第一塊磁碟到虛擬機器rac1-12c,命令如下:

VBoxManage storageattach rac1-12c --storagectl "SATA" --port 4 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar1.vdi --mtype shareable

圖中虛擬機器名稱:

埠號指的是:

  1. VBoxManage storageattach rac1-12c --storagectl "SATA" --port 4 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar1.vdi --mtype shareable

  2. VBoxManage storageattach rac1-12c --storagectl "SATA" --port 5 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar2.vdi --mtype shareable

  3. VBoxManage storageattach rac1-12c --storagectl "SATA" --port 6 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar3.vdi --mtype shareable

  4. VBoxManage storageattach rac1-12c --storagectl "SATA" --port 7 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar4.vdi --mtype shareable

  5. VBoxManage storageattach rac1-12c --storagectl "SATA" --port 8 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar5.vdi --mtype shareable

  6. VBoxManage storageattach rac1-12c --storagectl "SATA" --port 9 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar6.vdi --mtype shareable

  7. VBoxManage storageattach rac1-12c --storagectl "SATA" --port 10 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar7.vdi --mtype shareable

將7塊磁碟全部連線至rac1-12c完成後,在圖形介面看到如下:

36、連線完成後,需要把7塊磁碟改成共享磁碟屬性。

  1. VBoxManage modifyhd E:\實驗環境\12CRAC\V_SHARES\taryartar1.vdi --type shareable

  2. VBoxManage modifyhd E:\實驗環境\12CRAC\V_SHARES\taryartar2.vdi --type shareable

  3. VBoxManage modifyhd E:\實驗環境\12CRAC\V_SHARES\taryartar3.vdi --type shareable

  4. VBoxManage modifyhd E:\實驗環境\12CRAC\V_SHARES\taryartar4.vdi --type shareable

  5. VBoxManage modifyhd E:\實驗環境\12CRAC\V_SHARES\taryartar5.vdi --type shareable

  6. VBoxManage modifyhd E:\實驗環境\12CRAC\V_SHARES\taryartar6.vdi --type shareable

  7. VBoxManage modifyhd E:\實驗環境\12CRAC\V_SHARES\taryartar7.vdi --type shareable

執行完成後,在把7塊盤都連線到另外3臺虛擬機器

37、指令碼如下:

  1. VBoxManage storageattach rac2-12c --storagectl "SATA" --port 4 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar1.vdi --mtype shareable

  2. VBoxManage storageattach rac2-12c --storagectl "SATA" --port 5 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar2.vdi --mtype shareable

  3. VBoxManage storageattach rac2-12c --storagectl "SATA" --port 6 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar3.vdi --mtype shareable

  4. VBoxManage storageattach rac2-12c --storagectl "SATA" --port 7 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar4.vdi --mtype shareable

  5. VBoxManage storageattach rac2-12c --storagectl "SATA" --port 8 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar5.vdi --mtype shareable

  6. VBoxManage storageattach rac2-12c --storagectl "SATA" --port 9 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar6.vdi --mtype shareable

  7. VBoxManage storageattach rac2-12c --storagectl "SATA" --port 10 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar7.vdi --mtype shareable

  8. VBoxManage storageattach rac3-12c --storagectl "SATA" --port 4 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar1.vdi --mtype shareable

  9. VBoxManage storageattach rac3-12c --storagectl "SATA" --port 5 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar2.vdi --mtype shareable

  10. VBoxManage storageattach rac3-12c --storagectl "SATA" --port 6 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar3.vdi --mtype shareable

  11. VBoxManage storageattach rac3-12c --storagectl "SATA" --port 7 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar4.vdi --mtype shareable

  12. VBoxManage storageattach rac3-12c --storagectl "SATA" --port 8 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar5.vdi --mtype shareable

  13. VBoxManage storageattach rac3-12c --storagectl "SATA" --port 9 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar6.vdi --mtype shareable

  14. VBoxManage storageattach rac3-12c --storagectl "SATA" --port 10 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar7.vdi --mtype shareable

  15. VBoxManage storageattach rac4-12c --storagectl "SATA" --port 4 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar1.vdi --mtype shareable

  16. VBoxManage storageattach rac4-12c --storagectl "SATA" --port 5 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar2.vdi --mtype shareable

  17. VBoxManage storageattach rac4-12c --storagectl "SATA" --port 6 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar3.vdi --mtype shareable

  18. VBoxManage storageattach rac4-12c --storagectl "SATA" --port 7 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar4.vdi --mtype shareable

  19. VBoxManage storageattach rac4-12c --storagectl "SATA" --port 8 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar5.vdi --mtype shareable

  20. VBoxManage storageattach rac4-12c --storagectl "SATA" --port 9 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar6.vdi --mtype shareable

  21. VBoxManage storageattach rac4-12c --storagectl "SATA" --port 10 --device 0 --type hdd --medium E:\實驗環境\12CRAC\V_SHARES\taryartar7.vdi --mtype shareable

38、共享儲存新增完成,啟動4臺虛擬機器,然後用crt登陸。

使用fdisk -l |grep Di檢視每臺主機的硬碟。

  1. [[email protected] ~]# fdisk -l|grep Di

  2. Disk /dev/sda: 32.2 GB, 32212254720 bytes

  3. Disk identifier: 0x000ca0fb

  4. Disk /dev/sdb: 12.9 GB, 12884901888 bytes

  5. Disk identifier: 0x00000000

  6. Disk /dev/sdc: 12.9 GB, 12884901888 bytes

  7. Disk identifier: 0x00000000

  8. Disk /dev/sdd: 2147 MB, 2147483648 bytes

  9. Disk identifier: 0x00000000

  10. Disk /dev/sde: 2147 MB, 2147483648 bytes

  11. Disk identifier: 0x00000000

  12. Disk /dev/sdf: 2147 MB, 2147483648 bytes

  13. Disk identifier: 0x00000000

  14. Disk /dev/sdg: 2147 MB, 2147483648 bytes

  15. Disk identifier: 0x00000000

  16. Disk /dev/sdh: 2147 MB, 2147483648 bytes

  17. Disk identifier: 0x00000000

  18. Disk /dev/mapper/vg_rac1-lv_root: 28.5 GB, 28529655808 bytes

  19. Disk identifier: 0x00000000

  20. Disk /dev/mapper/vg_rac1-lv_swap: 3154 MB, 3154116608 bytes

  21. Disk identifier: 0x00000000

四臺虛擬機器都顯示如上,表示一切正常。

sda是虛擬機器本地普通磁碟。從sd[b-h]是新新增的磁碟。其中sd[b-c]是兩塊10g的共享盤,後面5塊sd[d-h]是2g的共享盤。


39、對7塊共享盤進行分割槽,每個磁碟只分一個區。注意共享盤分割槽只需要在一個節點上做就可以了,做完其它節點就能看到新的分割槽 。

在節點rac1上進行分割槽操作。

如下:

http://blog.csdn.net/kadwf123/article/details/78244863

分割槽完成後,在所有的節點執行fdisk -l

可以看到sd[b-h]都已經增加了一個分割槽了,如下我在節點1下分割槽,我截取了節點2看到的分割槽情況:

  1. [[email protected] ~]# fdisk -l

  2. Disk /dev/sda: 32.2 GB, 32212254720 bytes

  3. 255 heads, 63 sectors/track, 3916 cylinders

  4. Units = cylinders of 16065 * 512 = 8225280 bytes

  5. Sector size (logical/physical): 512 bytes / 512 bytes

  6. I/O size (minimum/optimal): 512 bytes / 512 bytes

  7. Disk identifier: 0x000ca0fb

  8. Device Boot Start End Blocks Id System

  9. /dev/sda1 * 1 64 512000 83 Linux

  10. Partition 1 does not end on cylinder boundary.

  11. /dev/sda2 64 3917 30944256 8e Linux LVM

  12. Disk /dev/sdb: 12.9 GB, 12884901888 bytes

  13. 255 heads, 63 sectors/track, 1566 cylinders

  14. Units = cylinders of 16065 * 512 = 8225280 bytes

  15. Sector size (logical/physical): 512 bytes / 512 bytes

  16. I/O size (minimum/optimal): 512 bytes / 512 bytes

  17. Disk identifier: 0x06b87c6d

  18. Device Boot Start End Blocks Id System

  19. /dev/sdb1 1 1566 12578863+ 83 Linux

  20. Disk /dev/sdc: 12.9 GB, 12884901888 bytes

  21. 255 heads, 63 sectors/track, 1566 cylinders

  22. Units = cylinders of 16065 * 512 = 8225280 bytes

  23. Sector size (logical/physical): 512 bytes / 512 bytes

  24. I/O size (minimum/optimal): 512 bytes / 512 bytes

  25. Disk identifier: 0x15863b60

  26. Device Boot Start End Blocks Id System

  27. /dev/sdc1 1 1566 12578863+ 83 Linux

  28. Disk /dev/sdd: 2147 MB, 2147483648 bytes

  29. 255 heads, 63 sectors/track, 261 cylinders

  30. Units = cylinders of 16065 * 512 = 8225280 bytes

  31. Sector size (logical/physical): 512 bytes / 512 bytes

  32. I/O size (minimum/optimal): 512 bytes / 512 bytes

  33. Disk identifier: 0x12264bdf

  34. Device Boot Start End Blocks Id System

  35. /dev/sdd1 1 261 2096451 83 Linux

  36. Disk /de