Packaging:Minutes20070904

Fedora Packaging Committee Meeting of {2007-09-04}

Present

• JasonTibbitts (tibbs)
• JesseKeating (f13)
• RalfCorsepius (racor)
• RexDieter (rdieter)
• ToshioKuratomi (abadger1999)
• VilleSkyttä (scop)

Writeups

The following drafts have been accepted by FESCO and are to be written into the guidelines:

• The first three sections of http://fedoraproject.org/wiki/PackagingDrafts/PHP (Requires and Provides for PEAR and PECL packages and Macros and Scriptlets for PECL packages).
• A clarification/rewording of the "File and Directory Ownership" guideline: https://www.redhat.com/archives/fedora-packaging/2007-August/msg00084.html

Votes

There were no votes this week.

Other Discussions

Guidelines for packaging Python eggs: https://fedoraproject.org/wiki/PackagingDrafts/PythonEggs

There is more work to be done here; if you are interested in Python module packaging please read over the draft and voice your concerns on the mailing list.

IRC Logs

[12:02]  <abadger1999> meetin' time?
[12:04]  <thm> [summertime :) ] 
[12:04]  <jeremy> abadger1999: what do you want to meet about? :)
[12:05]  <warren> jeremy, today's meeting is to plan tomorrow's meeting.
[12:05]  <abadger1999> Packaging Committee :-)
[12:06]  --> racor has joined this channel (n=rc040203@HSI-KBW-082-212-056-027.hsi.kabelbw.de).
[12:06]  <abadger1999> Python Eggs, new initscripts, that kind of stuff.
[12:07]  <tibbs> I'm not sure we have a quorum at this point.
[12:08]  <tibbs> Is spot around?
[12:08]  <jeremy> he was taking the day off, so don't think so
[12:09]  <scop> I count 6
[12:09]  <scop> (w/o spot)
[12:09]  <tibbs> Well, PHP guideline tweaks were approved; he's listed as taking care of those.
[12:10]  <tibbs> Otherwise Eggs are on the schedule.
[12:11]  <abadger1999> Eggs, isn't quite ready yet.  But I'd like some feedback from people.
[12:11]  <abadger1999> jeremy: You could give input on this as well.
[12:11]  <abadger1999> Two issues that might be controversial:
[12:11]  <abadger1999> 1) Should we make eggs for packages which don't have them upstream?
[12:12]  <abadger1999> This has come up with trac and docutils.
[12:12]  <abadger1999> Trac uses docutils if it's present.  Unfortunately it can't tell if it's present if docutils is not installed as an egg.
[12:13]  <thm> which doesn't interfere with being installed normally
[12:13]  <abadger1999> Debian Unstable has recently switched to installing egg info for docutils despite it not having that upstream.
[12:14]  <abadger1999> thm: Right.  Being installed as an egg doesn't interfere with import docutils; it just provides additional methods for a project to tell that another module is installed.
[12:14]  <jeremy> abadger1999: I started to look at what you wrote up one day last week, but then got distracted fixing something for test2 instead
[12:15]  <scop> are there any drawbacks if we just do eggs when needed even if upstream doesn't?
[12:16]  <scop> I don't know much at all about eggs, but somehow this sounds to me just like whether we should be allowed to add *.pc files if upstream doesn't or not
[12:16]  <tibbs> What does the %files section look like when you package up an old version?  The examples only include %build and %install.
[12:16]  <thm> encouraging people to rely on that info, even when not available upstream
[12:17]  <abadger1999> scop: AFAICT the two drawbacks are: Won't be consistent across distros (looks like it isn't consistent already)
[12:17]  <tibbs> thm: Could you identify yourself for the meeting minutes?
[12:17]  <abadger1999> scop: And we'll either have to patch setup.py to include dependency information or ship eggs that don't identify their dependencies.
[12:18]  <scop> ok, so it sounds pretty much like adding *.pc indeed
[12:18]  <abadger1999> tibbs: There will be an egg/egg-info directory in %{python_sitelib} as well as the normal MODULENAME directory
[12:19]  <abadger1999> scop: Yes.  That sounds like a good comparison.
[12:19]  <scop> is there a reason why some upstreams don't/wouldn't accept patches that egg-enable their stuff?
[12:19]  <tibbs> So the modulename directory is still unversioned?
[12:19]  <thm> tibbs: sorry, will stay quiet now. I reported bz 256541.
[12:19]  <abadger1999> tibbs: That actually ties into the second issue.
[12:20]  <abadger1999> scop: I think a *very* few projects don't like setuptools on principle.
[12:22]  <tibbs> thm: You're welcome to participate if you have something to add.  From https://bugzilla.redhat.com/show_bug.cgi?id=256541 it's obvious that you know far more about the issue than many of us do.
[12:22]  <thm> scop: in case of docutils, one reason seems to be that they want to stay compatible with Python < 2.3
[12:23]  <abadger1999> scop: For docutils, upstream doesn't want to limit the versions of python that docutils can run on just because of the build script... setuptools requires (I believe) python2.3
[12:23]  <scop> ok
[12:23]  <scop> there's no "compatiblity mode" or something for earlier versions?
[12:24]  <abadger1999> Hmmm.... "for Python 2.3.5 and up on most platforms; 64-bit platforms require a minimum of Python 2.4"
[12:24]  <abadger1999> No compat-mode that I know of.
[12:25]  * f13 looks in.
[12:27]  <tibbs> I think the bottom line is that we should do whatever is necessary to make our own packaging reasonable and self-consistent.
[12:27]  <tibbs> But what was the second issue?
[12:27]  <abadger1999> Second issue is multiple versions.
[12:28]  <abadger1999> It's what motivated me to look into python eggs in the first place but it's turning out to be a bit of a pain.
[12:29]  <abadger1999> I worked out something that nearly worked in the draft guidelines but when I talked to setuptools upstream about the final problems he had some issues with what we're doing.
[12:30]  <abadger1999> My goals were: 1) import MODULE should import whatever version we decide is the default
[12:30]  <abadger1999> 2) setup.py modified for setuptools should work to select a specific version from the eggs we've installed.
[12:31]  <abadger1999> 3) In a simple script that doesn't use setup.py there should be an API to select a specific version from the installed eggs.
[12:31]  <abadger1999> #3 is where upstream has a problem.
[12:31]  <tibbs> Who is "upstream" in this case?
[12:32]  <abadger1999> Philip J Eby
[12:32]  <abadger1999> setuptools author
[12:32]  <tibbs> Oh, the setuptool upstream.  OK.
[12:32]  <tibbs> But why would they care about cases where their software isn't being used?
[12:33]  <abadger1999> Well, we need to select the version someway.  There is a pkg_resource module that setuptools provides to let us interact with that egg information.
[12:34]  <abadger1999> In particular, there's a documented function pkg_resources.require('MODULE(><=)VERSION') that works as long as there's no default version
[12:34]  <abadger1999> ie: #1 would not work.
[12:35]  <thm> we could have a wrapper module for #1
[12:35]  <abadger1999> And there's an undocumented variable __requires__='MODULE(><=)VERSION' that can be set before importing modules.
[12:35]  <scop> hm, from 2) and 3) - if using setup.py, is the version that will be used by our package decided at build time?
[12:35]  <abadger1999> (With import pkg_resources being the first module to import after setting __require__)
[12:36]  <abadger1999> scop: Sort of.  The version is defined in setup.py and that definition is carried over to the egginfo (in the requires.txt file)
[12:36]  <-- racor has left this server ("Leaving").
[12:37]  <abadger1999> So setup.py might have: install_requires= ['SQLAlchemy>=0.3,<0.4', 'CherryPy>=2.0,<3.0'] 
[12:37]  <abadger1999> The installed package will have a requires.txt with:
[12:37]  <abadger1999> SQLAlchemy>=0.3,<0.4
[12:37]  <abadger1999> CherryPy>=2.0,<3.0
[12:38]  <abadger1999> scop: So I guess the answer is no, the decision is still made at run time.
[12:39]  <scop> ok, so the versions of SQLAlchemy and CherryPy that were available at the time the package was built don't end up as eg. CherryPy=2.5 somewhere
[12:39]  <abadger1999> Correct.
[12:39]  <scop> good
[12:40]  <abadger1999> getting back to the issue: __requires__ works but Philip refuses to document it and says it's not going to be a supported way of switching versions.
[12:40]  <abadger1999> So although it works where pkg_resources does not, it leaves us on shaky ground if we want to tell people to use it.
[12:41]  <abadger1999> So how important is that third goal for us?  Do we have to have a way for people to easily select between versions in the interpreter shell or for quick and dirty scripting?
[12:42]  <scop> does PYTHONPATH in environment not work for that?
[12:43]  <scop> I understand it probably isn't quite a complete solution to what you were looking for in #3
[12:44]  <abadger1999> It's not ideal because setuptools will override PYTHONPATH.  So you won't be able to try out pkg_resources while also relying on PYTHONPATH to give you some eggs.
[12:44]  <abadger1999> You'll have to use one or the other.
[12:45]  <scop> doh
[12:45]  <abadger1999> Also, it leaves you to figure out the paths, like: PYTHONPATH=/usr/lib/python2.5/site-packages/SQLAlchemy-0.3.10-py2.5.egg/sqlalchemy/:$PYTHONPATH
[12:46]  <abadger1999> and update that if sqlalchemy 0.3.11 ever comes out.
[12:46]  <abadger1999> Rather than a nice API like:   __requires__='SQLAlchemy>=0.3,<0.4alpha'; import pkg_resources; import sqlalchemy
[12:47]  <scop> yep, understood
[12:47]  <abadger1999> Finally, I'll have to check whether PYTHONPATH can point to zipped up modules (an egg is a zip just like a jar in java)
[12:49]  <abadger1999> If it can't we'd have to make sure the guidelines specify --no-zip whenever installing eggs.
[12:50]  <thm> abadger1999: additionally some packages are not safe to be used in unzipped form, afaik
[12:50]  <thm> oops ... in zipped form
[12:51]  <abadger1999> thm: I thought it was the other way around.  Some weren't safe to use in zipped form.
[12:51]  <abadger1999> :-)
[12:52]  <abadger1999> setuptools is pretty conservative about installing a module zipped but we might want to specify --no-zip in relation to installing eggs where upstream does not ship it.
[12:53]  <thm> right. setuptools uses heuristics to decide that
[12:53]  <abadger1999> So the question is: How much does having an API matter?
[12:54]  <abadger1999> This is python; I'm used to having an API for everything :-)
[12:54]  <scop> can't tell, but if upstream has made it clear that __requires__ is unsupported and undocumented and will remain that way, I don't think it makes sense for us to recommend its usage
[12:58]  <abadger1999> Right.  So I guess another way of saying this is: Do we use eggs for managing multiple versions, use something of our own design, or write an alternative to pkg_resources that has an API that we can use?
[12:59]  <abadger1999> I think I lean towards the first with the idea that if the feature is needed enough, someone (maybe us, maybe another distro) will do the third and we can pick it up then.
[13:00]  <scop> abadger1999, +1
[13:01]  <tibbs> Well, I guess that's the end of the packaging meeting.
[13:01]  <abadger1999> Thanks for letting htis turn into the "python eggs meeting" :-)
[13:02]  <scop> np, I think we're done
[13:02]  <tibbs> abadger1999: Did you have anything else?
[13:02]  <abadger1999> Nope.  If you have feedback later today, just send mail to the list.
[13:02]  <tibbs> OK, I guess that's it.
[13:02]  <abadger1999> I'll have a revision out before the next meeting.
[13:03]  <scop> abadger1999, thm, thanks for the info on eggs and your patience :)