CRS-1714 Unable to discover any voting files

APPLIES TO

Operating System - Oracle Linux Server 7.9
Oracle Database - Enterprise Edition - Version 11.2.0.4.0
Oracle Grid Infrastructure - Version 11.2.0.4.0

SYMPTOMS

操作系统重启后,crs 无法正常启动

[grid@rac1 ~]$ crsctl stat res -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.

查看 /u01/app/11.2.0/grid/log/rac1/alertrac1.log 日志,出现无法识别到 voting files 报错

[cssd(3348)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
2022-05-01 12:51:13.898:
[cssd(3348)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
2022-05-01 12:51:28.914:
[cssd(3348)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log

检查 oracleasm configure,已经设置开机启动扫描磁盘

[root@rac1 ~]# oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_SCAN_DIRECTORIES=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"

而且手动扫描磁盘是可以扫到磁盘的,说明配置是没啥问题的

[root@rac1 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "OCR_VOTE1"
Instantiating disk "OCR_VOTE2"
Instantiating disk "OCR_VOTE3"

CAUSE

由于磁盘通过 iSCSI 共享映射的,经过排查是因为 oracleasm.service 的服务比 iscsid.service 的服务更早启动,导致 oracleasm 没有扫描到 iSCSI 共享磁盘

SOLUTION

修改一下服务让 oracleasm 在 iscsid 后面启动

[grid@rac1 ~]$ cat /usr/lib/systemd/system/oracleasm.service
[Unit]
Description=Load oracleasm Modules
Requires=iscsid.service # add
After=iscsi.service iscsid.service # add

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/oracleasm.init start_sysctl
ExecStop=/usr/sbin/oracleasm.init stop_sysctl
ExecReload=/usr/sbin/oracleasm.init restart_sysctl

[Install]
WantedBy=multi-user.target

重启后,问题解决。