Make sure you click watch on your new page so that you are notified of changes to it by others, including the Feature Wrangler
Debug Python stacks
Summary
Fedora now ships debug versions of Python 2 and Python 3 in addition to the traditional optimized builds. This will be of use to advanced Python users, such as developers of extension modules.
Owner
- Name: Dave Malcolm
- Email: <dmalcolm@redhat.com>
Current status
- Targeted release: Fedora 42
- Last updated: 2010-05-20
- Percentage of completion: 10%
Initial notes on this: DaveMalcolm/PythonIdeas
See email
See the upstream notes on what all the flags do
| Package | Latest build | Debug flags |
|---|---|---|
| python | python-2.6.5-9.fc14 | --with-py-debug (implies Py_DEBUG, which implies LLTRACE, Py_REF_DEBUG, Py_TRACE_REFS, and PYMALLOC_DEBUG)
|
| python3 | python3-3.1.2-6.fc14 | --with-py-debug (implies Py_DEBUG, which implies LLTRACE, Py_REF_DEBUG, Py_TRACE_REFS, and PYMALLOC_DEBUG)
|
- Other flags to investigate:
- COUNT_ALLOCS
- CALL_PROFILE
- WITH_TSC
- Figure out sane RPM conventions for packaging debug builds of extension modules
- Package debug builds of important extension modules
Detailed Description
In previous releases we have configured our build of Python for the typical use-case: as much optimization as reasonable.
However, upstream Python supports a number of useful debug options which use more RAM and CPU cycles, but make it easier to track down bugs [2]
Typically these are of use to people working on Python C extensions, for example, for tracking down awkward reference-counting mistakes.
In Fedora 14 we now supply a python-debug package containing a debug build of Python with these settings turned on.
It is intended for use by advanced Python users, and is installable on top of the normal (optimized) build. The builds share the same .py and .pyc files, but have their own compiled libraries and extension modules.
Technical notes
The Fedora 14 python.src.rpm now configures and builds, and installs the python sources twice, once with the regular optimized settings, and again with debug settings. (in most cases the files are identical between the two installs, and for the files that are different, they get separate paths)
The builds are set up so that they can share the same .py and .pyc files - they have the same bytecode format.
However, they are incompatible at the machine-code level: the extra debug-checking options change the layout of Python objects in memory, so the configurations have different shared library ABIs. A compiled C extension built for one will not work with the other.
The key to keeping the different module ABIs separate is that module "foo.so" for the standard optimized build will instead be "foo_d.so i.e. gaining a "_d" suffix to the filename, and this is what the "import" routine will look for. This convention ultimately comes from the way the Windows build is set up in the upstream build process, via a similar patch that Debian apply.
Similarly, the optimized libpython2.6.so.1.0 now has a libpython2.6_d.so.1.0 cousin for the debug build: all of the extension modules are linked against the appropriate libpython, and there's a /usr/include/python2.6-debug directory, parallel with the /usr/include/python2.6 directory. There's a new "sys.pydebug" boolean to distinguish the two configurations, and the distutils module uses this to supply the appropriate header paths ,and linker flags when building C extension modules.
Finally, the debug build's python binary is /usr/bin/python2.6-debug, hardlinked as /usr/bin/python-debug (as opposed to /usr/bin/python2.6 and /usr/bin/python)
Benefit to Fedora
Scope
How To Test
Verify optimized python stacks
FIXME
Verify debug python stacks
FIXME
Verify Py_REF_DEBUG
Install python-debug and python3-debug
$ python-debug -c "import sys; print(sys.gettotalrefcount())" 28564 [15039 refs] $ python3-debug -c "import sys; print(sys.gettotalrefcount())" 28564 [15039 refs]
and ensure that each prints a number to stdout (and a refcount to stderr)
Verify Py_TRACE_REFS
Verify that python-debug can print all live objects:
$ python-debug -c "import sys; print(sys.getobjects(0))" | less [<frame object at 0x1181eb0>, <code object <module> at 0x7f379227f970, file "<string>", line 1>, (-1, None, 0), (snip) $ python3-debug -c "import sys; print(sys.getobjects(0))" | less [b'<string>', <frame object at 0xd78060>, <code object <module> at 0x7f84cc5ddbf0, file "<string>", line 1>, (0, None), (snip)
In both cases there ought to be a large amount of debug information sent to stdout
Verify that python-debug can print all live objects of a given type (e.g. "int"):
$ python-debug -c "import sys; print(sys.getobjects(0, int))" |less [8000, 512, 2147483647, 590923713, 907133923, (snip) $ python3-debug -c "import sys; print(sys.getobjects(0, int))" [131180, 327681, 327680, 131119, 131121, 131120, 131118, 131116, (snip)
Verify that setting the PYTHONDUMPREFS environment variable causes lots of info to be dumped to stderr on exit:
$ PYTHONDUMPREFS=1 python-debug -c "pass" [15039 refs] Remaining objects: 0x7fba34c1ac08 [1] 'last_traceback' 0x7fba34c1aba0 [1] 'last_value' 0x7fba34c1a860 [1] 'last_type' (etc) $ PYTHONDUMPREFS=1 python3-debug -c "pass" [35078 refs] Remaining objects: 0x20c4148 [1] b'flush' 0x1bf1640 [1] b'OverflowError' 0x1c55860 [1] b'UnboundLocalError' (etc)
Verify PYMALLOC_DEBUG
Verify the PYTHONMALLOCSTATS environment variable.
Ensure that running with the env var set causes debugging information to be logged to stderr at exit:
$ PYTHONMALLOCSTATS=1 python-debug -c "pass"
Small block threshold = 256, in 32 size classes.
class size num pools blocks in use avail blocks
----- ---- --------- ------------- ------------
4 40 1 1 100
6 56 1 1 71
7 64 1 30 33
9 80 1 1 49
10 88 1 2 44
11 96 29 560 658
12 104 21 394 404
13 112 21 418 338
14 120 45 1032 453
15 128 37 943 204
16 136 10 182 108
17 144 5 82 58
18 152 3 37 41
19 160 3 21 54
20 168 26 313 311
21 176 2 14 32
22 184 1 5 17
23 192 2 5 37
24 200 2 4 36
25 208 1 6 13
26 216 1 1 17
27 224 1 2 16
28 232 2 6 28
29 240 1 2 14
30 248 1 2 14
31 256 1 1 14
# times object malloc called = 58,737
# arenas allocated total = 5
# arenas reclaimed = 0
# arenas highwater mark = 5
# arenas allocated current = 5
5 arenas * 262144 bytes/arena = 1,310,720
# bytes in allocated blocks = 496,176
# bytes in available blocks = 381,488
95 unused pools * 4096 bytes = 389,120
# bytes lost to pool headers = 10,560
# bytes lost to quantization = 12,896
# bytes lost to arena alignment = 20,480
Total = 1,310,720
$ PYTHONMALLOCSTATS=1 python3-debug -c "pass"
FIXME: can we verify the buffer overrun code?
Verify COUNT_ALLOCS
python-debug -c "import sys; from pprint import pprint ; pprint(sys.getcounts())" python3-debug -c "import sys; from pprint import pprint ; pprint(sys.getcounts())"
Verify LLTRACE
python-debug -c "__lltrace__ = True ; import site" python3-debug -c "__lltrace__ = True ; import site"
Verify CALL_PROFILE
FIXME
Verify WITH_TSC
FIXME
python-debug -c "import sys ; sys.settscdump(True) ; print(42)" python3-debug -c "import sys ; sys.settscdump(True) ; print(42)"
User Experience
Fedora 14 now has a python-debug package containing debug versions of all of the content of the regular subpackages emitted by the python build (as opposed to the python-debuginfo package, which contains data for use by gdb (and thus is of use by the optimized stack).
The optimized build should be unaffected by the presence (or availability) of the debug build: all of the paths and the ELF metadata for the standard build should be unchanged compared to how they were before adding the debug configuration.
Installing the debug package gives you a /usr/bin/python-debug, analogous to the regular /usr/bin/python
The interactive mode of this version tells you the total reference count of all live Python objects after each command:
[david@fedora14 devel]$ python-debug Python 2.6.5 (r265:79063, May 19 2010, 18:20:14) [GCC 4.4.3 20100422 (Red Hat 4.4.3-18)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print "hello world" hello world [28748 refs] >>> [28748 refs] [15041 refs]
The debug build shares most of the files with the regular build (.py/.pyc/.pyo files; directories; support data; documentation); the only differences are the ELF files (binaries/shared libraries), and infrastructure relating to configuration (Include files, Makefile, python-config => python-debug-config, etc) that are different.
Dependencies
None
Contingency Plan
FIXME
Documentation
Release Notes
- FIXME