DaveMalcolm/PythonIdeas

Multiple Python configurations

What might we want to vary?

"Debug" versus "standard" build

Python can be built with various debugging options, typically just used when working on Python itself. In my experience "--with-pydebug" roughly halves the speed of the interpreter, but adds lots of extra checking (e.g. reference leaks). It also changes the ABI.

We could produce an entirely separate "debug" python stack, with various debugging checks configured in (breaking ABI, and slowing things down); hope was to make it trivial to run /usr/bin/python2.6-debug (provided you also have the debug build of all of the modules, etc; doesn't work, you need -debug of full stack, I think; you'd have a yum-debug, with -debug of all the stack).

Configuration option	Definition	Effect
--with-pydebug	Py_DEBUG	Implies ABI-breaking other options (see below), and adds additional self-tests throughout libpython
	Py_TRACE_REFS	(implied by Py_DEBUG): Track a doubly-linked list of all objects. Changes libpython ABI: adds _ob_prev and _ob_next members to PyObject
	Py_REF_DEBUG	(implied by Py_TRACE_REFS); track total reference count of all objects (_Py_RefTotal), and ensure it never goes negative
--without-pymalloc		Avoid dealing with the arena allocator: go straight to malloc/free; changes API meaning
	COUNT_ALLOCS	Track the number of allocations made per PyTypeObject; changes API meaning
--with-tsc	WITH_TSC	enable/disable timestamp counter profile
	DYNAMIC_EXECUTION_PROFILE	Track how many times each opcode is executed
	DXPAIRS	Track how many times each pair of opcodes is executed
	LLTRACE	Track all stack operations
	USE_STACKCHECK

(See pyconfig.h)

--with-valgrind (not needed if --without-pymalloc)

--enable-profiling ?

Future work: we may want to have with/without LLVM as a build option

"--with-pydebug" is the big one, implying various ABI-breaking changes.

Might want to turn off optimizations, or only optimize lightly, whilst we're at it (e.g. "-Os" ?)

Would need to do as a Feature. Do it for python3 as well, potentially giving 4 subpackages per build:

  python2
  python2-pydebug
  python3
  python3-pydebug

UCS2 vs UCS4

Is it sane to double the number of configurations, offering UCS2 vs UCS4? I don't think so, but listing it here for completeness. I'm not convinced that the choice of UCS4 is correct; are we merely doing it due to historical precedent?

Multiple minor versions

- advance versions of 2.7 and 3.2

Alternate Implementations

Can we use the following to build out multiple stacks? (e.g. share pure-python modules between Jython and CPython?)

• Unladen Swallow
• Jython
• PyPy
  • http://pypy.org/download.html
  • https://codespeak.net/issue/pypy-dev/issue432 SELinux issue

Dealing with multiple python implementations

What would packaging guidelines look like? Generalization of current guidelines to add additional builds. See my notes sent to the packaging list before (and roundly rejected) Too much work for package maintainers?

Other things we could change

dist-packages rather than site-packages ?

Debian uses "dist-packages" for things coming as part of the distribution, to avoid PyPi users splatting "site-packages"; should we? It seems more logical to me, keeping site-packages for site-specific use.