Highly Available IP (HAIP) on Oracle 11.2.0.2

在 Oracle 11.2.0.2 之前，私网的冗余一般是通过在 OS 上做网卡绑定（如 Bond 等）来实现的，从 Oracle 11.2.0.2 版本开始推出 HAIP（Highly Available Virtual IP）技术替代了操作系统层面的网卡绑定技术，功能更强大、更兼容。HAIP 通过其提供的独特的 169.254.* 网段的 IP 地址实现集群内部链接的高可用及负载均衡。所以，在 11.2.0.2 或更高版本安装 RAC 的时候需要注意 169.254.* 的 IP 地址不能被占用。有了 HAIP 技术则可以不再需要借助任何第三方的冗余技术来实现私网网卡的冗余。

不建议禁用 HAIP，后续维护可能有坑。

HAIP default parameters

# ora.cluster_interconnect.haip
~]# $GRID_HOME/bin/crsctl stat res ora.cluster_interconnect.haip -p -init | grep ENABLED
ENABLED=1

# ora.asm
~]# $GRID_HOME/bin/crsctl stat res ora.asm -p -init | grep START_DEPENDENCIES
START_DEPENDENCIES=hard(ora.cssd,ora.cluster_interconnect.haip,ora.ctssd)pullup(ora.cssd,ora.cluster_interconnect.haip,ora.ctssd)weak(ora.drivers.acfs)

~]# $GRID_HOME/bin/crsctl stat res ora.asm -p -init | grep STOP_DEPENDENCIES
STOP_DEPENDENCIES=hard(intermediate:ora.cssd,shutdown:ora.cluster_interconnect.haip)

Disable HAIP

# 1. stop crs on all nodes
~]# $GRID_HOME/bin/crsctl stop crs

# 2. modify res attribute on all nodes

~]# $GRID_HOME/bin/crsctl start crs -excl -nocrs
~]# $GRID_HOME/bin/crsctl stop res ora.asm -init
~]# $GRID_HOME/bin/crsctl modify res ora.cluster_interconnect.haip -attr "ENABLED=0" -init
~]# $GRID_HOME/bin/crsctl modify res ora.asm -attr "START_DEPENDENCIES='hard(ora.cssd,ora.ctssd)pullup(ora.cssd,ora.ctssd)weak(ora.drivers.acfs)',STOP_DEPENDENCIES='hard(intermediate:ora.cssd)'" -init

~]# $GRID_HOME/bin/crsctl stop crs

# 3. start crs on all nodes

~]# $GRID_HOME/bin/crsctl start crs

# 4. check the haip disabled

~]# $GRID_HOME/bin/crsctl stat res -init -t | grep -A1 haip
ora.cluster_interconnect.haip
      1        ONLINE  OFFLINE

~]# ip a | grep 169.254

~]# $GRID_HOME/bin/crsctl stat res ora.cluster_interconnect.haip -p -init | grep ENABLED
ENABLED=0

~]# $GRID_HOME/bin/crsctl stat res ora.asm -p -init | grep START_DEPENDENCIES
START_DEPENDENCIES=hard(ora.cssd,ora.ctssd)pullup(ora.cssd,ora.ctssd)weak(ora.drivers.acfs)

~]# $GRID_HOME/bin/crsctl stat res ora.asm -p -init | grep STOP_DEPENDENCIES
STOP_DEPENDENCIES=hard(intermediate:ora.cssd)

Set environment variable export HAIP_UNSUPPORTED=TRUE before run $GRID_HOME/root.sh script to disable HAIP feature, but not work while adding node to exist cluster.

Enable HAIP

~]# $GRID_HOME/bin/crsctl modify res ora.cluster_interconnect.haip -attr "ENABLED=1" -init

~]# $GRID_HOME/bin/crsctl modify res ora.asm -attr "START_DEPENDENCIES='hard(ora.cssd,ora.cluster_interconnect.haip,ora.ctssd)pullup(ora.cssd,ora.cluster_interconnect.haip,ora.ctssd)weak(ora.drivers.acfs)',STOP_DEPENDENCIES='hard(intermediate:ora.cssd,shutdown:ora.cluster_interconnect.haip)'" -init