Xorg Debugging
Did you know, that all of the modular X packages now come with full debuginfo packages, including the X server and driver modules?
The X server uses dlopen now also, so it can be debugged using the stock gdb that comes with Fedora.
In order to debug the running X server, install the debuginfo packages for it, and any drivers you are loading. You then need to have 2 computers[1] networked together, or using a serial port or some other remote access method.
Edit the xorg.conf file, and enable the option:
Option "NoTrapSignals"
Then, enable coredumps on the system, if you have not done so already, by using "ulimit -c unlimited" in a script dropped into /etc/profile.d/enable-coredumps.sh or whatever. Be sure to run the script or log out and back in.
As the X server is a setuid root process, and the kernel will not allow coredumps of SUID processes by default, you have 2 choices.
You can tweak the kernel to permit SUID coredumps by doing:
echo 1 > /proc/sys/kernel/core_setuid_ok
or you can run the X server as root. If you enable the kernel to allow SUID coredumps as per above, just be sure to disable it afterwards by echoing '0' to the same file, to avoid potential security issues.
Now you are ready to break things and debug them. I prefer to start the X server with "startx" all the time, but particularly when debugging. I recommend doing the same, unless the problem you are debugging only occurs when using gdm/kdm/xdm, as it tends to be much simpler to debug with startx. YMMV however.
If the server crashes, you can gdb the coredump now. Assuming you installed debuginfo packages for the server (make sure the debuginfo package is the same version-release as the server that is installed), you should get useful backtraces, etc.
If you want to debug the running server, single step it, etc., you need the second computer[1] mentioned above. Do all the same steps as above, but this time, ssh in from the remote computer (or use minicom on serial port or whatever), and get the PID of the X server with 'pidof X'. Then attach gdb to the X server:
gdb --pid=$(pidof X)
Now go to town! Find the bug, install the X server/driver sources, fix the bug, generate a patch with gendiff, and attach it to a bug report. It's really that simple!!! ;oP
This informative message has been brought to you by the Red Hat X Development team, and Frank's Red Hot Chile & Lime Hot Sauce.
Have fun!
[1] The reason that 2 computers are required, is that if you start
the X server, open an xterm and run gdb on the running X server,
gdb will stop the X server, and try to print to stdout. Since
stdout is your xterm, and xterm is displaying to the X server which
is now no longer responding because gdb stopped it, you are screwed.
It's kindof like trying to stand on a carpet, and then pull it out
from under yourself. You can't really do it. This is why you
need[2] 2 computers.
[2] Ok, it is technically possible to do it with one computer too, but it is complex enough to not really bother trying to do, so don't even waste time thinking of it. If you don't have 2 computers, go dumpster diving at 3am in the garbage dumpster of a computer store. They throw out good stuff I'm told, and you can have a "X server debugging terminal" in no time, like everyone else on the block. Alternatively, surf ebay.com for a bargain, or hang out at a computer recycling shop.