Revision as of 18:22, 27 February 2009

Proposal for package Ruby Gem including extension library written in C

Current status

Current packaging guidelines on Fedora about Ruby Gems are mainly written on https://fedoraproject.org/wiki/Packaging/Ruby#Ruby_Gems

The guideline says:

The install should be performed with the command
$ gem install --local --install-dir %{buildroot}%{gemdir} --force %{SOURCE0}

Problem

This causes some problems especially when Ruby Gem contains extension libraries written in C because of the following reason.

A-1

First of all the above procedure will cause the error like following.

+ /usr/lib/rpm/check-buildroot
Binary file /home/tasaka1/rpmbuild/INSTROOT/rubygem-zoom-0.4.1-4.fc9p-root-tasaka1/usr/lib/debug/usr/lib/ruby/site_ruby/1.8/i386-linux/zoom.so.debug matches
Found '/home/tasaka1/rpmbuild/INSTROOT/rubygem-zoom-0.4.1-4.fc9p-root-tasaka1' in installed files; aborting
error: Bad exit status from /home/tasaka1/rpmbuild/INSTROOT/rpm-tmp.84251 (%install)

This is because with above command:

gem first unpacks files in the gem file under %{buildroot}%{gemdir}
Then gem (actually gcc) compiles C source files installed under there.

So the rebuilt C library may contain %{buildroot} string.

A-2

So to avoid this error some maintainers on Fedora of rubygems including extension written in C simply strip compiled C libraries before %install stage ends.

Of course this won't create debuginfo file correctly.
Also, when Gem is directly installed under %{buildroot} and %prep is blank, find_debuginfo.sh won't be called anyway because %{buildsubdir} is not defined (this is because debugXXX.list can't be written when %{_builddir}/%{buildsubdir} is missing).

Solution to A

The problem written in A can be resolved when Ruby Gems are installed under %{_builddir} first.

B

Currently there is no mention about how to pass Fedora specific compilation flags correctly.

Solution to B

Usually ruby extension uses ruby script file named extconf.rb which has require 'mkmf'. mkmf.rb module accepts CONFIGURE_ARGS environment to specify CFLAGS so using this is preferred.

even for non-Gem Ruby packages, many packages containing C shared libraries
- have extconf.rb in the source archives
- or have Rakefile which create extconf.rb

which usually contain require mkmf. In such case

export CONFIGURE_ARGS="--with-cflags='%{optflags}'"

can be used, too. So using this method is preferred even for non-Gem packages.

A package which uses extconf.rb directly can take --with-cflags= argument directly, like ruby extconf.rb --with-cflags="%{optflags}", however in the case that Rakefile creates extconf.rb, usually rake build does not accept --with-cflags= argument.

Proposition

Add the following sentences at the end of Ruby packages with binary content/shared libraries

For packages which create C shared libraries using extconf.rb

export CONFIGURE_ARGS="--with-cflags='%{optflags}'"

should be used to pass CFLAGS to Makefile correctly. This also applies to Ruby Gems.

And add below.

Ruby Gem with extension libraries written in C

When Ruby Gem contains extension libraries written in C,

First %{_builddir}/%{buildsubdir} must be created at %prep stage.
Then the Ruby Gem must be installed under %{_builddir}/%{buildsubdir} at %build stage to get C libraries compiled under the directory.
When using gem command to install Gem file, using -V option is recommend to check if CFLAGS is correctly honored.
Finally at %install stage the whole tree under %{_builddir}/%{buildsubdir}%{gemdir} should be copied (not moved) to under %{buildroot}%{gemdir}.
- When all tree under %{_builddir}/%{buildsubdir} is moved to under %{buildroot}, find_debuginfo.sh will complain that the corresponding source files are missing.
Installed C codes (usually under %{geminstdir}/etc) may be removed even if gem contents %{gemname} reports that installed C codes should be found there.

Revised proposition

From the comments I received, it seems that the following proposition is clearer.

(The part CONFIGURE_ARGS=... stays unchanged)

Ruby Gem with extension libraries written in C

When Ruby Gem contains extension libraries written in C,

First, %prep stage must contain %setup -q -c -T to create the directory where C libraries are compiled.
Then at %build stage the Ruby Gem must be installed under the directory created at %prep stage to get C libraries compiled under there.
When gem install is used to install Gem file, using -V option is recommend to check if CFLAGS is correctly honored.
Finally at %install stage the whole tree under the directory created at %prep stage should be copied (not moved) to under %{buildroot}%{gemdir}.
- When all tree under the directory created at %prep stage is moved to under %{buildroot}, find_debuginfo.sh will complain that the corresponding source files are missing.
Installed C codes (usually under %{geminstdir}/etc) may be removed even if gem contents %{gemname} reports that installed C codes should be found there.

Note

The current guideline

If the Gem contains binary content (e.g., for a database driver), it must be marked 
as architecture specific, and all architecture specific content must be moved 
from the %{gemdir} to the [#ruby_sitearch %{ruby_sitearch} directory] during %install

must still apply.

Example

%prep
%setup -q -T -c

%build
mkdir -p ./%{gemdir}
export CONFIGURE_ARGS="--with-cflags='%{optflags}'"
gem install --local --install-dir ./%{gemdir} -V --force %{SOURCE0}

%install
rm -rf %{buildroot}
mkdir -p %{buildroot}%{gemdir}
cp -a ./%{gemdir}/* %{buildroot}%{gemdir}

mkdir -p %{buildroot}%{ruby_sitearch}
mv %{buildroot}%{geminstdir}/lib/*.so %{buildroot}%{ruby_sitearch}
rm -rf %{buildroot}%{geminstdir}/ext

%clean
rm -rf %{buildroot}

%files
%defattr(-,root,root,-)
%{ruby_sitearch}/*.so
%{geminstdir}/
%{gemdir}/cache/%{gemname}-%{version}.gem
%{gemdir}/specifications/%{gemname}-%{version}.gemspec

Issues

[SOLVED] Installed C codes

Currently all C codes (in Gem file) are also installed with gem install (usually under %{geminstdir}/ext). In my recognition these files are used only for creating C libraries and are not needed anymore. So actually some packagers simply remove these files.

Note that when also non-Gem type of source archive is provided, when using non-Gem version such C codes are usually not installed.

However Gem has its own metadata and Gem actually reports that these files should have been installed.

Example
- Currently in rubygem-pam rpm no files are under %{geminstdir}/ext (on i386 this is currently /usr/lib/ruby/gems/1.8/gems/pam-1.5.3/ext).
- However gem contents pam actually returns some files under this directory.

How should we treat these files?

simply don't remove these files
ignore gem report in this case and remove these anyway
seperate these files into subpackages (e.g. -source subpackage)
or anything else

[SOLVED] To pass CFLAGS correctly

I noticed that even for non-Gem Ruby packages, many packages containing C shared libraries

have extconf.rb in the source archives
or have Rakefile which create extconf.rb