1. 问题现象
1.1. 两台 RAC 节点的 OS 时间差超过 5 分钟
发现两个节点的时间确实相差较大
[oraprod@ebsdb1 ~]$ date
Wed Apr 30 11:27:20 CST 2014
[oraprod@ebsdb2 ~]$ date
Wed Apr 30 11:22:24 CST 2014

1.2. 检查 CTSS 状态发现其处于观察模式
[oraprod@ebsdb1 bin]$ crsctl check ctss
CRS-4700: The Cluster Time Synchronization Service is in Observer mode.
[oraprod@ebsdb1 bin]$ cluvfy comp clocksync
Verifying Clock Synchronization across the cluster nodes
Checking if Clusterware is installed on all nodes...
Check of Clusterware install passed
Checking if CTSS Resource is running on all nodes...
CTSS resource check passed
Querying CTSS for time offset on all nodes...
Query of CTSS for time offset passed
Check CTSS state started...
CTSS is in Observer state. Switching over to clock synchronization checks using NTP
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
NTP Configuration file check passed
Checking daemon liveness...
Liveness check failed for "ntpd"
Check failed on nodes:
ebsdb1
PRVF-5494 : The NTP Daemon or Service was not alive on all nodes
PRVF-5415 : Check to see if NTP daemon or service is running failed
Clock synchronization check using Network Time Protocol(NTP) failed 文章版权所有 Jusin Hao（luckyfriends） ，支持原创，转载请注明。
PRVF-9652 : Cluster Time Synchronization Services check failed
Verification of Clock Synchronization across the cluster nodes was unsuccessful on all the specified nodes.
1.3. 发现 RAC 节点的 NTP 配置存在
发现RAC2 上的NTP 是起着的，RAC1 上的NTP 是听着的
[root@ebsdb2 ~]# service ntpd status
ntpd (pid 7723) is running...
[root@ebsdb2 ~]# cd /etc/
[root@ebsdb2 etc]# ls -alt |grep ntp
-rw-r--r-- 1 root root 1833 Apr 16 2013 ntp.conf
-rw-r--r-- 1 root root 1833 Nov 17 2011 ntp.conf.original
drwxr-xr-x 2 root root 4096 Jun 11 2011 ntp
[root@ebsdb2 ntp]# cd /var/run/
[root@ebsdb2 run]# ls -alt |grep ntp
-rw-r--r-- 1 root root 4 Apr 17 2013 ntpd.pid
[root@ebsdb1 ~]# service ntpd status
ntpd is stopped
[root@ebsdb1 ~]# cd /etc/
[root@ebsdb1 etc]# ls -alt ntp*
-rw-r--r-- 1 root root 1833 Apr 16 2013 ntp.conf

2. 问题解决过程：
2.1. 停止应用
略。
2.2. 停止 Oracle RAC 及数据库
[oraprod@ebsdb1 bin]$ ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.CRS.dg ora....up.type ONLINE ONLINE ebsdb1
ora.DATA.dg ora....up.type ONLINE ONLINE ebsdb1
ora.FRA.dg ora....up.type ONLINE ONLINE ebsdb1
ora....ER.lsnr ora....er.type ONLINE ONLINE ebsdb1
ora....N1.lsnr ora....er.type ONLINE ONLINE ebsdb1
ora.asm ora.asm.type ONLINE ONLINE ebsdb1
ora.cvu ora.cvu.type ONLINE ONLINE ebsdb1
ora....SM1.asm application ONLINE ONLINE ebsdb1
ora....B1.lsnr application ONLINE ONLINE ebsdb1
ora.ebsdb1.gsd application ONLINE OFFLINE
ora.ebsdb1.ons application ONLINE ONLINE ebsdb1
ora.ebsdb1.vip ora....t1.type ONLINE ONLINE ebsdb1
ora....SM2.asm application ONLINE ONLINE ebsdb2
ora....B2.lsnr application ONLINE ONLINE ebsdb2
ora.ebsdb2.gsd application ONLINE OFFLINE
ora.ebsdb2.ons application ONLINE ONLINE ebsdb2
ora.ebsdb2.vip ora....t1.type ONLINE ONLINE ebsdb2
ora.gsd ora.gsd.type ONLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE ebsdb1
ora.oc4j ora.oc4j.type ONLINE ONLINE ebsdb1
ora.ons ora.ons.type ONLINE ONLINE ebsdb1
ora.prod.db ora....se.type ONLINE ONLINE ebsdb2
ora.scan1.vip ora....ip.type ONLINE ONLINE ebsdb1

[oraprod@ebsdb1 bin]$ ./crs_stop -all
Attempting to stop `ora.ons` on member `ebsdb2`
Attempting to stop `ora.LISTENER.lsnr` on member `ebsdb2`
Attempting to stop `ora.cvu` on member `ebsdb1`
Attempting to stop `ora.ons` on member `ebsdb1`
Attempting to stop `ora.LISTENER.lsnr` on member `ebsdb1`
Attempting to stop `ora.LISTENER_SCAN1.lsnr` on member `ebsdb1`
Attempting to stop `ora.oc4j` on member `ebsdb1`
Attempting to stop `ora.CRS.dg` on member `ebsdb2`
Attempting to stop `ora.DATA.dg` on member `ebsdb2`
Attempting to stop `ora.FRA.dg` on member `ebsdb2`
Attempting to stop `ora.prod.db` on member `ebsdb2`
CRS-2789: Cannot stop resource 'ora.gsd' as it is not running on server 'ebsdb2'
Attempting to stop `ora.CRS.dg` on member `ebsdb1`
Attempting to stop `ora.DATA.dg` on member `ebsdb1`
Attempting to stop `ora.FRA.dg` on member `ebsdb1`
Attempting to stop `ora.prod.db` on member `ebsdb1`
CRS-2789: Cannot stop resource 'ora.gsd' as it is not running on server 'ebsdb1'
Stop of `ora.cvu` on member `ebsdb1` succeeded.

Stop of `ora.LISTENER.lsnr` on member `ebsdb1` succeeded.
Attempting to stop `ora.ebsdb1.vip` on member `ebsdb1`
CRS-2789: Cannot stop resource 'ora.LISTENER.lsnr' as it is not running on server 'ebsdb1'
Stop of `ora.LISTENER_SCAN1.lsnr` on member `ebsdb1` succeeded.
Attempting to stop `ora.scan1.vip` on member `ebsdb1`
Stop of `ora.ons` on member `ebsdb1` succeeded.
CRS-2789: Cannot stop resource 'ora.ons' as it is not running on server 'ebsdb1'
Stop of `ora.LISTENER.lsnr` on member `ebsdb2` succeeded.
Attempting to stop `ora.ebsdb2.vip` on member `ebsdb2`
CRS-2789: Cannot stop resource 'ora.LISTENER.lsnr' as it is not running on server 'ebsdb2'
Stop of `ora.ons` on member `ebsdb2` succeeded.
CRS-2789: Cannot stop resource 'ora.ons' as it is not running on server 'ebsdb2'
Stop of `ora.scan1.vip` on member `ebsdb1` succeeded.
Stop of `ora.ebsdb1.vip` on member `ebsdb1` succeeded.
Attempting to stop `ora.net1.network` on member `ebsdb1`
Stop of `ora.net1.network` on member `ebsdb1` succeeded.
Stop of `ora.ebsdb2.vip` on member `ebsdb2` succeeded.
Attempting to stop `ora.net1.network` on member `ebsdb2`
Stop of `ora.net1.network` on member `ebsdb2` succeeded.
Stop of `ora.FRA.dg` on member `ebsdb2` succeeded.
Stop of `ora.FRA.dg` on member `ebsdb1` succeeded.
Stop of `ora.oc4j` on member `ebsdb1` succeeded.
Stop of `ora.DATA.dg` on member `ebsdb1` succeeded.
Stop of `ora.DATA.dg` on member `ebsdb2` succeeded.
Stop of `ora.prod.db` on member `ebsdb2` succeeded.
Stop of `ora.CRS.dg` on member `ebsdb1` succeeded.
Stop of `ora.CRS.dg` on member `ebsdb2` succeeded.
Attempting to stop `ora.asm` on member `ebsdb2`
Stop of `ora.prod.db` on member `ebsdb1` succeeded.
Attempting to stop `ora.asm` on member `ebsdb1`
Stop of `ora.asm` on member `ebsdb2` succeeded.------在这里长时间等待停止
Attempting to stop `ora.asm` on member `ebsdb2`
CRS-2500: Cannot stop resource 'ora.asm' as it is not running
Stop of `ora.asm` on member `ebsdb2` succeeded.
Stop of `ora.asm` on member `ebsdb1` succeeded.
Attempting to stop `ora.asm` on member `ebsdb1`
CRS-0184: Cannot communicate with the CRS daemon.

2.3. ASM 在 CRS 停止前无法停止
Attempting to stop `ora.asm` on member `ebsdb1`
Stop of `ora.asm` on member `ebsdb2` succeeded.------在这里长时间等待停止
Attempting to stop `ora.asm` on member `ebsdb2`
[oraprod@ebsdb1 bin]$ ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[oraprod@ebsdb1 bin]$ crsctl check ctss
CRS-4700: The Cluster Time Synchronization Service is in Observer mode.
查看asm 日志
[oraprod@ebsdb1 trace]$ tail -f alert_+ASM1.log
SUCCESS: ALTER DISKGROUP DATA DISMOUNT /* asm agent *//* {1:52692:21791} */
freeing rdom 2
Received detach msg from inst 2 for dom 2
GMON querying group 1 at 23 for pid 18, osid 8802
Sun May 11 09:26:24 2014
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 1
Sun May 11 09:26:24 2014
ORA-15032: not all alterations performed
ORA-15027: active use of diskgroup "CRS" precludes（妨碍、组织） its dismount
ERROR: ALTER DISKGROUP CRS DISMOUNT /* asm agent *//* {1:52692:21791} */
从日志可以看出ASM无法 dismount 因为 CRS 没有停止，这是因为 The OCR is located on ASM and CRSD is client of
ASM thus ASM can't be shutdown（参考 ORA-15097 Cannot Shutdown ASM if OCR is Located in ASM [ID 984663.1]）
即在 Oracle 11gR2 下，除了数据文件之外，OCR 也是放在 ASM 里的，这样 CRSD 就会成为 ASM 的客户端。如果
我们像 Oracle 10g 里那样直接停ASM 的时候，就会报上面的错误。 所以，在 11gR2 下面，要停 ASM 实例的话，
只能和 CRS 一起停。 在root 用户下执行：crsctl stop crs。
同时，要注意不要直接 kill 掉 ASM 进程，或者用 shutdown abort 来关闭ASM 实例。 这样 CRS 也会被挂掉。

2.4. 停止 CRS
[root@ebsdb1 ~]# cd /oradb/11.2.0/grid/bin/
[root@ebsdb1 bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ebsdb1'
CRS-2673: Attempting to stop 'ora.crsd' on 'ebsdb1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'ebsdb1'

CRS-4549: Unexpected disconnect while executing shutdown request.
CRS-2675: Stop of 'ora.crsd' on 'ebsdb1' failed
CRS-2679: Attempting to clean 'ora.crsd' on 'ebsdb1'
CRS-2681: Clean of 'ora.crsd' on 'ebsdb1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'ebsdb1'
CRS-2673: Attempting to stop 'ora.crf' on 'ebsdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'ebsdb1'
CRS-2673: Attempting to stop 'ora.evmd' on 'ebsdb1'
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ebsdb1'
CRS-2677: Stop of 'ora.mdnsd' on 'ebsdb1' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ebsdb1' succeeded
CRS-2677: Stop of 'ora.crf' on 'ebsdb1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'ebsdb1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'ebsdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'ebsdb1'
CRS-2677: Stop of 'ora.cssd' on 'ebsdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'ebsdb1'
CRS-2677: Stop of 'ora.gipcd' on 'ebsdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'ebsdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'ebsdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ebsdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@ebsdb2 bin]# ./crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ebsdb2'
CRS-2673: Attempting to stop 'ora.crf' on 'ebsdb2'
CRS-2673: Attempting to stop 'ora.ctssd' on 'ebsdb2'
CRS-2673: Attempting to stop 'ora.evmd' on 'ebsdb2'
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ebsdb2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'ebsdb2'
CRS-2677: Stop of 'ora.crf' on 'ebsdb2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'ebsdb2' succeeded
CRS-2677: Stop of 'ora.evmd' on 'ebsdb2' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ebsdb2' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'ebsdb2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'ebsdb2'
CRS-2677: Stop of 'ora.cssd' on 'ebsdb2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'ebsdb2'
CRS-2677: Stop of 'ora.gipcd' on 'ebsdb2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'ebsdb2'
CRS-2677: Stop of 'ora.gpnpd' on 'ebsdb2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ebsdb2' has completed
[oraprod@ebsdb1 bin]$
[oraprod@ebsdb1 bin]$ ./crs_stat -t
CRS-0184: Cannot communicate with the CRS daemon.
[oraprod@ebsdb1 bin]$ ps -ef |grep asm
oraprod 30959 28971 0 09:37 pts/1 00:00:00 grep asm
[oraprod@ebsdb1 bin]$ ps -ef |grep d.bin
oraprod 30971 28971 0 09:37 pts/1 00:00:00 grep d.bin

2.5. 调整 RAC 服务器时间
2.5.1. 同步服务器的 OS 时间与 CMOS 时间
[root@ebsdb1 ~]# date
Sun May 11 09:41:57 CST 2014
[root@ebsdb1 ~]# hwclock ----查看 rac1 的 cmos 时间
Sun 11 May 2014 09:36:34 AM CST -0.670362 seconds
[root@ebsdb2 bin]# date
Sun May 11 09:35:57 CST 2014
[root@ebsdb2 bin]# hwclock ----查看 rac2 的 cmos 时间
Sun 11 May 2014 09:36:20 AM CST -0.662735 seconds
[root@ebsdb1 ~]# clock –systohc ----同步 cmos 时间与 os 时间
[root@ebsdb1 ~]# hwclock
dateSun 11 May 2014 09:43:17 AM CST -0.983102 seconds
[root@ebsdb1 ~]# date
Sun May 11 09:43:17 CST 2014

2.5.2. 修改 EBSDB2（RAC2）时间与 RAC1 一致
[root@ebsdb1 run]# date
Sun May 11 09:54:06 CST 2014
[root@ebsdb1 run]# hwclock
Sun 11 May 2014 09:54:09 AM CST -0.920398 seconds
[root@ebsdb2 run]# date
Sun May 11 09:46:33 CST 2014
[root@ebsdb2 run]# date -s 09:54:09 ----修改 RAC2 的时间与RAC1 一致
Sun May 11 09:54:09 CST 2014
[root@ebsdb2 run]# date
Sun May 11 09:54:14 CST 2014
[root@ebsdb2 run]# hwclock -r
Sun May 11 09:48:15 2014 -0.045481 seconds
[root@ebsdb2 run]# clock -w -----将时间写入 CMOS
[root@ebsdb2 run]# hwclock
Sun May 11 09:55:48 2014 -0.686071 seconds
[root@ebsdb2 run]# hwclock -r
Sun May 11 09:55:53 2014 -1.014941 seconds
[root@ebsdb2 run]# date
Sun May 11 09:55:55 CST 2014

2.6. 停止 OS 的 NTPD 并禁用
2.6.1. 停止 NTPD 服务
[root@ebsdb1 ~]# service ntpd status
ntpd is stopped
[root@ebsdb2 bin]# service ntpd status
ntpd is stopped
[root@ebsdb2 bin]# ps -ef |grep ntpd
root 21397 20896 0 09:40 pts/2 00:00:00 grep ntpd
2.6.2. 禁用 NTPD 自动启动
[root@ebsdb1 ~]# chkconfig ntpd off
[root@ebsdb2 ~]# chkconfig ntpd off
2.6.3. 重命名 ntpd.conf
[root@ebsdb1 etc]# ls -alt |grep ntp
-rw-r--r-- 1 root root 1833 Apr 16 2013 ntp.conf
-rw-r--r-- 1 root root 1833 Nov 17 2011 ntp.conf.original
drwxr-xr-x 2 root root 4096 Jun 11 2011 ntp
[root@ebsdb1 etc]# mv /etc/ntp.conf /etc/ntp.conf.20140511
[root@ebsdb1 etc]# ls -alt |grep ntp
-rw-r--r-- 1 root root 1833 Apr 16 2013 ntp.conf.20140511
-rw-r--r-- 1 root root 1833 Nov 17 2011 ntp.conf.original
drwxr-xr-x 2 root root 4096 Jun 11 2011 ntp
[root@ebsdb2 etc]# ls -alt |grep ntp
-rw-r--r-- 1 root root 1833 Apr 16 2013 ntp.conf
-rw-r--r-- 1 root root 1833 Nov 17 2011 ntp.conf.original
drwxr-xr-x 2 root root 4096 Jun 11 2011 ntp
[root@ebsdb2 etc]# mv ntp.conf ntp.conf.20140511
[root@ebsdb2 etc]# ls -alt |grep ntp
-rw-r--r-- 1 root root 1833 Apr 16 2013 ntp.conf.20140511
-rw-r--r-- 1 root root 1833 Nov 17 2011 ntp.conf.original
drwxr-xr-x 2 root root 4096 Jun 11 2011 ntp
2.6.4. 删除 PID 文件
[root@ebsdb1 etc]# cd /var/run
[root@ebsdb1 run]# ls -alt |grep ntp
[root@ebsdb2 etc]# cd /var/run
[root@ebsdb2 run]# ls -alt |grep ntp
2.7. 启动 Oracle RAC
[root@ebsdb1 run]# ps -ef |grep d.bin
root 32003 29334 0 09:58 pts/2 00:00:00 grep d.bin
[root@ebsdb2 run]# ps -ef |grep d.bin
root 21985 20896 0 09:58 pts/2 00:00:00 grep d.bin
2.7.1. 启动 RAC1 的 crs
[root@ebsdb1 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@ebsdb1 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
[root@ebsdb1 bin]# ps -ef |grep d.bin
oraprod 354 1 4 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/oraagent.bin
oraprod 357 1 1 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/scriptagent.bin
root 361 1 2 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/orarootagent.bin
root 610 29334 0 10:01 pts/2 00:00:00 grep d.bin
root 32057 1 2 09:59 ? 00:00:02 /oradb/11.2.0/grid/bin/ohasd.bin reboot
oraprod 32173 1 0 09:59 ? 00:00:00 /oradb/11.2.0/grid/bin/oraagent.bin
oraprod 32185 1 0 09:59 ? 00:00:00 /oradb/11.2.0/grid/bin/mdnsd.bin
oraprod 32195 1 0 09:59 ? 00:00:00 /oradb/11.2.0/grid/bin/gpnpd.bin
root 32206 1 1 09:59 ? 00:00:00 /oradb/11.2.0/grid/bin/orarootagent.bin
oraprod 32208 1 1 09:59 ? 00:00:00 /oradb/11.2.0/grid/bin/gipcd.bin
root 32223 1 8 09:59 ? 00:00:05 /oradb/11.2.0/grid/bin/osysmond.bin
root 32234 1 0 09:59 ? 00:00:00 /oradb/11.2.0/grid/bin/cssdmonitor
root 32252 1 0 09:59 ? 00:00:00 /oradb/11.2.0/grid/bin/cssdagent
oraprod 32264 1 2 09:59 ? 00:00:01 /oradb/11.2.0/grid/bin/ocssd.bin
root 32295 1 1 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/ologgerd -M -d
/oradb/11.2.0/grid/crf/db/ebsdb1
root 32341 1 0 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/octssd.bin reboot
oraprod 32363 1 0 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/evmd.bin
root 32661 1 11 10:00 ? 00:00:01 /oradb/11.2.0/grid/bin/crsd.bin reboot
oraprod 32731 32363 0 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/evmlogger.bin -o
/oradb/11.2.0/grid/evm/log/evmlogger.info -l /oradb/11.2.0/grid/evm/log/evmlogger.log
2.7.2. 启动 RAC2 的 crs
[root@ebsdb2 bin]# ./crsctl check crs
CRS-4639: Could not contact Oracle High Availability Services
[root@ebsdb2 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@ebsdb2 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
[root@ebsdb2 bin]# ps -ef |grep d.bin
root 22148 1 4 10:00 ? 00:00:02 /oradb/11.2.0/grid/bin/ohasd.bin reboot
oraprod 22265 1 0 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/oraagent.bin
oraprod 22277 1 0 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/mdnsd.bin
oraprod 22287 1 1 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/gpnpd.bin
root 22299 1 1 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/orarootagent.bin
oraprod 22300 1 3 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/gipcd.bin
root 22315 1 6 10:00 ? 00:00:01 /oradb/11.2.0/grid/bin/osysmond.bin
root 22326 1 0 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/cssdmonitor
root 22343 1 0 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/cssdagent
oraprod 22355 1 3 10:00 ? 00:00:00 /oradb/11.2.0/grid/bin/ocssd.bin
root 22386 1 3 10:01 ? 00:00:00 /oradb/11.2.0/grid/bin/ologgerd -m ebsdb1 -r -d
/oradb/11.2.0/grid/crf/db/ebsdb2
root 22432 1 2 10:01 ? 00:00:00 /oradb/11.2.0/grid/bin/octssd.bin reboot
root 22449 20896 0 10:01 pts/2 00:00:00 grep d.bin
2.7.3. 查看 CTSS 状态已经出入 Active 模式
[root@ebsdb2 bin]# ./crsctl check ctss
CRS-4701: The Cluster Time Synchronization Service is in Active mode.
CRS-4702: Offset (in msec): 0
[root@ebsdb2 bin]#
[root@ebsdb1 bin]# ./crsctl check ctss
CRS-4701: The Cluster Time Synchronization Service is in Active mode.
CRS-4702: Offset (in msec): 0
[root@ebsdb1 bin]#
[root@ebsdb1 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@ebsdb2 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@ebsdb2 bin]# ./crsctl stat resource -t -init
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE ebsdb2 Started
ora.cluster_interconnect.haip
1 ONLINE ONLINE ebsdb2
ora.crf
1 ONLINE ONLINE ebsdb2
ora.crsd
1 ONLINE ONLINE ebsdb2
ora.cssd
1 ONLINE ONLINE ebsdb2
ora.cssdmonitor
1 ONLINE ONLINE ebsdb2
ora.ctssd
1 ONLINE ONLINE ebsdb2 ACTIVE:0
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE ebsdb2
ora.gipcd
1 ONLINE ONLINE ebsdb2
ora.gpnpd
1 ONLINE ONLINE ebsdb2
ora.mdnsd
1 ONLINE ONLINE ebsdb2
2.7.4. 查看集群状态
[root@ebsdb1 bin]# ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.CRS.dg ora....up.type ONLINE ONLINE ebsdb1
ora.DATA.dg ora....up.type ONLINE ONLINE ebsdb1
ora.FRA.dg ora....up.type ONLINE ONLINE ebsdb1
ora....ER.lsnr ora....er.type OFFLINE OFFLINE
ora....N1.lsnr ora....er.type OFFLINE OFFLINE
ora.asm ora.asm.type ONLINE ONLINE ebsdb1
ora.cvu ora.cvu.type OFFLINE OFFLINE
ora....SM1.asm application ONLINE ONLINE ebsdb1
ora....B1.lsnr application OFFLINE OFFLINE
ora.ebsdb1.gsd application ONLINE OFFLINE
ora.ebsdb1.ons application ONLINE ONLINE ebsdb1
ora.ebsdb1.vip ora....t1.type OFFLINE OFFLINE
ora....SM2.asm application ONLINE ONLINE ebsdb2
ora....B2.lsnr application OFFLINE OFFLINE
ora.ebsdb2.gsd application ONLINE OFFLINE
ora.ebsdb2.ons application ONLINE ONLINE ebsdb2
ora.ebsdb2.vip ora....t1.type OFFLINE OFFLINE
ora.gsd ora.gsd.type ONLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE ebsdb1
ora.oc4j ora.oc4j.type OFFLINE OFFLINE
ora.ons ora.ons.type ONLINE ONLINE ebsdb1
ora.prod.db ora....se.type OFFLINE OFFLINE
ora.scan1.vip ora....ip.type OFFLINE OFFLINE