Oracle Exadata Service Manual

X7/X8/X9

Replacement: Storage Cell Server F640 Flash Card

・ How to Replace an ExadataZDLRA X7,X8,X9 Storage Cell Server F640 Flash Card (Doc ID 2362260.1).pdf |

根据更换手册，该部件可以热拔插更换，但是实际操作中，热拔插可能会导致存储节点异常重启或 IO 异常，所以建议停机更换。

更换记录：

  1. 维护前准备工作。

    a) 记录故障存储节点基本信息。

    ~]# ipmitool sunoem cli "show /SYS product_serial_number"
    ~]# ipmitool sunoem cli "show /SP/network"
    ~]# ipmitool sunoem cli "show faulty"

    b) 确认故障存储节点前面板 DO NOT SERVICE 灯未亮起。

    ~]# cellcli -e list cell detail | grep -i donotservice

    c) 确认故障存储节点闪存卡状态为 dropped for replacement，记录 PCI 插槽位置。

    ~]# imageinfo
    ~]# cellcli -e list physicaldisk detail | egrep -i 'flash|slot|dropped'
    ~]# cellcli -e list physicaldisk <name> detail  # such as FLASH_10_1 / FLASH_10_2

    否则需要手动 Drop 故障闪存卡。

    d) 登录任一计算节点，检查 ASM 磁盘维护时间。

    ~]$ sqlplus / as sysasm
    SQL> select dg.name,a.value from v$asm_diskgroup dg, v$asm_attribute a where dg.group_number=a.group_number and a.name='disk_repair_time';  -- default 3.6H
    SQL> ALTER DISKGROUP DATA SET ATTRIBUTE 'DISK_REPAIR_TIME'='8.5H';

  2. 关闭故障存储节点。

    ~]# cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome  # All disks must show asmdeactivationoutcome=Yes

    ~]# cellcli -e alter griddisk all inactive  # Wait 10 minutes or longer
    ~]# cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome  # All disks must show asmmodestatus=OFFLINE/UNUSED and asmdeactivationoutcome=Yes
    ~]# cellcli -e list griddisk  # All grid disks show inactive

    ~]# cellcli -e alter cell shutdown services all
    ~]# shutdown -h now

  3. 拔掉电源线，拉出设备，打开盖板，佩戴防静电手环，更换闪存卡。

  4. 重新接入电源，登录 iLOM 检查硬件，启动操作系统。

    a) 检查是否还存在硬件故障。

    -> start /SP/faultmgmt/shell
    Are you sure you want to start /SP/faultmgmt/shell (y/n)? y

    faultmgmtsp> fmadm faulty
    No faults found
    faultmgmtsp> exit

    如果存在硬件问题，可尝试使用 fmadm repaired 命令修复。

    faultmgmtsp> fmadm repaire <uuid>

    b) 确认无硬件问题后，启动操作系统。

    -> start /SYS
    Are you sure you want to start /SYS (y/n)? y
    Starting /SYS

    -> start /HOST/console/
    Are you sure you want to start /HOST/console (y/n)? y
    Serial console started.  To stop, type ESC (

  5. 进入系统，检查磁盘状态，激活 griddisk。

    ~]# cellcli -e list cell detail
    ~]# cellcli -e list physicaldisk detail | egrep 'FLASH_|diskType|slot|status'  # return status: normal
    ~]# cellcli -e list physicaldisk <name> detail  # return status: normal
    ~]# cellcli -e list lun where disktype=flashdisk

    ~]# cellcli -e alter griddisk all active
    ~]# cellcli -e list griddisk  # All grid disks show active
    ~]# cellcli -e list griddisk attributes name,status,asmmodestatus,asmdeactivationoutcome  # All grid disks show asmmodestatus=ONLINE

    ~]# /opt/oracle.SupportTools/CheckHWnFWProfile -c strict
    ~]# /opt/oracle.SupportTools/ibdiagtools/verify-topology