KGDB Version 2

KGDB is a source level debugger for Linux kernel. It allows kernel developers to use GDB to debug the kernel in a similar way to the way application programs can be debugged.

This kernel Debugger version is not distributed as a kernel patches but it is already included in the last kernel tree released.

For more information concerning KGDB please click here

Index

• Major KGDB 2 features
• KGDB documentation
• Requirements
• Preparing the kernel
• Configuring KGDB via command line arguments
• Using GDB on the development machine
• Controlling the kernel execution
• Debugging the kernel
• Debugging the modules
• Known limits

Major KGDB 2 features

• KGDB can wait for connection from remote GDB much earlier than previous versions of KGDB.
• Now it is possible to detach GDB from a running kernel and reattach it. If GDB dies or has to be killed because of some problem, a new instance of GDB can be started and reconnected to KGDB.
• Support for ASC port: it will allow use of ASC (used in a lot of STM platforms) as debug port.
• Debug console ethernet support: (still experimental development). It allows using the ethernet as GDB/KGDB communication medium instead of a serial connection.

[Top]

KGDB documentation

The KGDB documentation is included in the kernel sources too. So you can have the KGDB internal documentation in different formats: sgmldocs (SGML), psdocs (Postscript), pdfdocs (PDF), htmldocs (HTML), mandocs (man pages)

For example executing the make pdfdocs command in the Documentation/DocBook directory will be created the kgdb.pfd file.

[Top]

Requirements

Hardware requirements

Two machines are required for using KGDB. The machine that runs the kernel to be debugged is called test machine (or simply target); the machine that runs GDB is called the development machine (or host).

Platforms supported

Currently KGDB 2 has been tested on the following STM platforms:

Software requirements

The development machine requires RedHat Enterprise Linux version 3 or later.

For the GDB front end, please see the section written in this page.

[Top]

Preparing the kernel

Configuring the kernel

Use the following command:

make ARCH=sh CROSS_COMPILE=sh4-linux- gconfig

To enable CONFIG_KGDB, enter under the "Kernel debugging" section and then select "KGDB: kernel debugging with remote gdb".

The following picture shows an example of KGDB configuration, in this case the debug port selected is the ttyAS1.

FIG 1: example of KGDB configuration

NOTE: while configuring, the help on line will give you more information concerning all KGDB options.

Compiling the kernel

Use the following command:

make ARCH=sh CROSS_COMPILE=sh4-linux- clean vmlinux

[Top]

Configuring KGDB via command line arguments

• Command line port options: after the kernel configuration phase you can change the KGDB port setup as shown bellow:
  • SH SCI(F) serial port:
    kgdbsci=0,115200
  • ST ASC port:
    kgdbasc=0,115200
  • Ethernet connection:
    kgdboe=[src-port]@/[dev],[tgt-port]@/[tgt-macaddr]
    where:
    • src-port (optional): source for UDP packets (defaults to 6443)
    • src-ip: source IP to use (interface address)
    • dev (optional): network interface (eth0)
    • tgt-port (optional): port GDB will use (defaults to 6442)
    • tgt-ip: IP address GDB will be connecting from
    • tgt-macaddr (optional): ethernet MAC address for logging agent (default is broadcast)
• kgdbwait: it makes KGDB wait for GDB connection during booting of the kernel. If defined it must be passed after the port options.

[Top]

Using GDB on the development machine

You must use the cross GDB installed with the EAR 2.0 distribution. You will find it in at following path /opt/STM/STLinux-2.0/devkit/sh4/bin/

Connecting GDB to the kernel debugger

On your development machine simply execute the following:

[dev_machine-#] sh4-linux-gdb vmlinux
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu --target=sh4-linux"...
Couldn't establish connection to remote target
(gdb) set remotebreak 1
(gdb) set remotebaud 115200
(gdb) target remote /dev/ttyS1
...

GDB front ends

GDB 6.3 provides a simple graphical interface. It may be useful to look at the source code during a debug section (doesn't allow you to set breakpoint etc.).

You can enable it using the "-" GDB command.

FIG 2: GDB 6.3 session

ddd (Data Display Debugger)

GNU ddd is a graphical front-end for GDB; for more information about it please click here.

To run the sh4-linux-gdb with ddd use the following command:

ddd --debugger sh4-linux-gdb vmlinux

Note: ddd must be used with --debugger sh4-linux-gdb option, infact, by default, it runs the host gdb while in this case it must run the cross gdb.

FIG 3: example: using ddd

ECLIPSE for ST40

You can debug the kernel by Eclipse, to do it you have to install the ST40 Linux Eclipse plugins.

For more information concerning Eclipse for Linux ST40 (and to download the plugin) see the following link.

[Top]

Controlling the kernel execution

Stopping the kernel execution

Magic SysRq support

It enables console device to interpret special characters as commands to dump state information or to invoke the kernel debugger.

To add this support in the kernel, enable the CONFIG_MAGIC_SYSRQ option during the configuration phase.

On your target you can break the kernel, starting a debug section, simply writing the letter 'g' in the /proc/sysrq-trigger file as shown below.

• On the target:

  root@target:# cat > /proc/sysrq-trigger SysRq : HELP : loglevel0-8 reBoot tErm Gdb kIll showMem showPc Sync showTasks Unmount g SysRq : GDB Entering GDB stub

• On the host: run the sh4-linux-gdb vmlinux command

Note: in the previous KGDB version 1 you could stop the kernel execution sending a break command ("Control-A F" using the minicom tool) and then the letter 'g' (within 5 seconds) in the system serial console. In the KGDB 2 this functionality has been removed.

Using Ctrl+C

To stop kernel execution, press Ctrl+C on the GDB terminal. So KGDB will take control of the kernel and contact GDB.

Note: if the GDB terminal doesn't respond to user input, you can close it sending an ASCII 3 character into the serial line as shown below.

echo -e "\003" > /dev/ttyS1

If the kernel was controlled by KGDB when GDB was killed, a new GDB can be started but all your breakpoints, before setting, will be lost.

Continuing the kernel execution

To continue kernel execution, use continue GDB command. GDB will tell the KGDB stub to continue kernel execution.

[Top]

Debugging the kernel

Using KGDB you can debug a kernel as a normal user application, for example you can place breakpoints, look at the stack and have information about the kernel threads.

Breakpoints

Use GDB break command to stop the kernel execution at a function or a source code line.

The break command can be accepted as argument:

• the function name
• the source code file name appended with ":" and the line number, as shown below:

  break drivers/char/tty_io.c:2887

Stepping through code

The GDB step command allows you to step a code statement. This command runs the kernel for one statement (entering into function calls too) and again hands over the control to GDB.

If you want to skip stepping into function calls use GDB next command.

Thread Analysis

GDB provides a listing of the kernel threads. So you can specify a particular thread to be analyzed and, using some GDB commands (like bt or info regi) get some information in context of the specified thread.

Useful GDB commands for debugging thread:

• info threads: display a summary of all threads currently in the kernel.

  GDB displays for each thread (in this order):

  1. the thread number assigned by GDB
  2. the target system thread identifier
  3. the current stack frame summary for that thread
  An asterisk *, placed on the left of the GDB thread number, indicates the current thread.

• thread threadno : to switch among threads.

Examining the Stack

You can look at the stack trace using the following GDB commands:

• backtrace (or bt): print a backtrace of the entire stack.

  Each line in the backtrace shows the frame number and the function name. The backtrace also shows the source file name and line number, as well as the arguments to the function.

• backtrace n (or bt n): print only the innermost n frames.
• backtrace -n (or bt -n): print only the outermost n frames.
• info stack (or info s): is aliases for backtrace.

[Top]

Debugging the modules

In this release GDB automatically is able to detect when a module has been loaded on the target machine so it can load the module object file into its memory for getting debugging information.

To let GDB know where module files are located, set the variable solib-search-path.

1. On the target: load your module

   root@target:# insmod my_mod.ko

2. Stop the kernel execution to confirm that the module is indeed loaded.

3. On the host: load the module symbols and set your breakpoints

   [dev_machine-#] sh4-linux-gdb vmlinux GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "--host=i686-pc-linux-gnu --target=sh4-linux"... 0x84031f10 in ?? () Automatically enabled KGDB extensions... (gdb) set solib-search-path /user/my_module_2.6/ Reading symbols from /user/my_module_2.6/my_mod.ko... expanding to full symbols...done. Loaded symbols for /user/my_module_2.6/my_mod.ko (gdb) info sharedlibrary From To Syms Read Shared Object Library 0xc0168000 0xc01680c0 Yes /user/my_module_2.6/my_mod.ko (gdb) b mod_stm_exit Breakpoint 1 at 0xc0168002: file /user/my_module_2.6/my_mod.c, line 43. (gdb) c Continuing.

Note: in order to understand where the lodable module symbols are located, GDB needs to be connected first to the target (where your modules have been loaded). This means that the command set solib-search-path /user/my_module_2.6/ must be run after the GDB has connected; otherwise it collects the symbols but it doesn't know where they are located. Similar issue may rise when you are debugging the module_exit function while threads, created by your module, are still runnig.

Configuring GDB to debug a module

To make easier your debugging activity, we suggest you to define your personal set of GDB commands in the .gdbinit file; this is done with the define GDB command.

Below a trivial example of .gdbinit file:

#### My command!
define mymod
  set solib-search-path /user/my_module_2.6/
  info sharedlibrary
end

#### Serial port setting
set remotebaud 115200
target remote /dev/ttyS0

As soon as GDB connects and you get the prompt just type the name of your command:

    ...
    (gdb) mymod

[Top]

Known limits

1. The hardware watchpoints/breakpoints are not yet supported.
2. The KGDB ethernet support has not been tested yet.
3. When the system console and the kernel debugger share the same serial port, please avoid executing another application (i.e. minicom) on that port. This could cause some serial communication errors.
4. When the system console and KGDB share the same serial port, if you omit the kgdbwait option the kernel boot phase will be stopped from KGDB and a GDB connection will be required (unfortunately no messages will be displayed). We assume that as a driver serial problem. At any rate, there is a very simple work around to avoid this unpleasant issue. During the kernel configuration phase, you can select for KGDB a unused system port; so you can boot the kernel without any problems and when you start a debug section you can reboot the kernel setting the debugger port options via command line.
5. Debugging a dynamic module we have a weakness. If you set breakpoints in a loadable module and at the end of your debugging section you unload the module (using rmmod linux command), without removing all the relevant breakpoints in GDB, as soon as you attempt to enter again into a debug section (for example stopping the kernel execution with Ctrl+C) the system hangs. In fact, KGDB is required to perform actions on the active breakpoints which are no longer accessible.
6. Performing the stack unwind, which is fairly complex task, involving GDB as well , you may have corrupted stack. To overcome this issue you may build the kernel and the relevant loadable modules in such away that the executing code uses the frame pointers. To do this set the compiling option in the "kernel hacking" section when you configure the kernel. This will imply the kernel to be compiled with the -fno-omit-frame-pointer flag.

[Top]