How to Analyze the Core Dump on Linux 7

Installing the crash utility

To analyze the vmcore dump file, you must have the crash and kernel-debuginfo packages installed.

~]# yum -y install yum-utils
~]# debuginfo-install kernel
~]# yum install crash

or install manually.

~]# rpm -ivh kernel-debuginfo-common-x86_64-3.10.0-1160.el7.x86_64.rpm
warning: kernel-debuginfo-common-x86_64-3.10.0-1160.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b6792c39: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:kernel-debuginfo-common-x86_64-3.################################# [100%]

~]# rpm -ivh /soft/kernel-debuginfo-3.10.0-1160.el7.x86_64.rpm
warning: /soft/kernel-debuginfo-3.10.0-1160.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b6792c39: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:kernel-debuginfo-3.10.0-1160.el7 ################################# [100%]

~]# yum install crash

Running the crash utility

Note: that the kernel version should be the same that was captured by kdump. To find out which kernel you are currently running, use the uname -r command.

~]# crash /usr/lib/debug/lib/modules/3.10.0-1160.el7.x86_64/vmlinux /var/crash/127.0.0.1-2024-09-09-11\:56\:41/vmcore

crash 7.2.3-11.el7
Copyright (C) 2002-2017 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: kernel relocated [130MB]: patching 87292 gdb minimal_symbol values

KERNEL: /usr/lib/debug/lib/modules/3.10.0-1160.el7.x86_64/vmlinux
DUMPFILE: /var/crash/127.0.0.1-2024-09-09-11:56:41/vmcore [PARTIAL DUMP]
CPUS: 4
DATE: Mon Sep 9 11:56:20 2024
UPTIME: 00:02:00
LOAD AVERAGE: 0.21, 0.17, 0.07
TASKS: 240
NODENAME: beautifly.localdomain
RELEASE: 3.10.0-1160.el7.x86_64
VERSION: #1 SMP Mon Oct 19 16:18:59 UTC 2020
MACHINE: x86_64 (2099 Mhz)
MEMORY: 8 GB
PANIC: "SysRq : Trigger a crash"
PID: 3998
COMMAND: "bash"
TASK: ffff9c9f7c383180 [THREAD_INFO: ffff9ca030efc000]
CPU: 3
STATE: TASK_RUNNING (SYSRQ)

crash> help

* extend log rd task
alias files mach repeat timer
ascii foreach mod runq tree
bpf fuser mount search union
bt gdb net set vm
btop help p sig vtop
dev ipcs ps struct waitq
dis irq pte swap whatis
eval kmem ptob sym wr
exit list ptov sys q

crash version: 7.2.3-11.el7 gdb version: 7.6
For help on any command above, enter "help <command>".
For help on input options, enter "help input".
For help on output options, enter "help output".

crash>