Perl/Tips

This page is intended to be a repository of "gotchas", and other little quick fixes, that aren't rare but are just uncommon enough that you forget how you did it last time:) This page is NOT part of the official packaging guidelines.

Module version dependencies too much specific

Rounding approach

When writing (Build)Requires, you can find the package requires or uses module Foo in very specific version (e.g. use Foo 0.2001;). If you just copy the version to spec file (Requires: perl(Foo) >= 0.2001) you can get unresolved depencecies because packaged perl-Foo's provide shorter version numbers (e.g. perl(Foo) = 0.16, perl(Foo) = 0.20, perl(Foo) = 0.30).

This is caused by the fact that Perl processes version strings as fractional numbers, but RPM as integers (e.g. RPM compares 0 to 0, and 2001 to 30).

There is no right solution. Current practice is to round dependency version up onto the the same number of digits as package version. (e.g. Requires: perl(Foo) >= 0.2001 becomes Requires: perl(Foo) >= 0.21). Of course this approach is meaningful only if current perl-Foo package has at least version 0.21.

If required package does not exist in requested version, the dependency package must be upgraded before (if upstream provides newer (e.g. version 0.30)). If highest upstream provides 0.2001 only, dependency package can be upgraded to provide perl(Foo) = 0.2001, however its maintainer must keep using augmented precision in future versions (e.g. instead of Provides: perl(Foo) = 0.30 she must write Provides: perl(Foo) = 0.3000) not to break RPM version comparison (each newer packager must have EVR string greater than previous one).

Dot approach

In the feature, one could consider different less error-prone approach: Instead of version rounding, one could transform each fraction version digit to next level version integer.

E.g. CPAN 12.34 became RPM 12.3.4. This method preserves ordering of fraction numbers, allows extending to more specific numbers and does not request package maintainer to remember number of augmented digits he needs to support in his SPEC file.

One must note that transition to this method can happen only at major version number change (the part before decimal dot) or at cost of new RPM epocha number.

See perl-Geo-IPfree.spec for live example.

Version contains underscore

Sometimes it is needed update to development release marked with something like _01. To be 100% sure, you should use in spec file:

%real_version 1.32_01
Version: 1.32.1

Beware underscores in code if not evaluated can produce warnings and program aborts in strict mode (examplary bug):

$VERSION = "1.32_01";

This must be solved by adding eval and reported to upstream as a bug (see Version Numbering in perlmodstyle(1)):

$VERSION = "1.32_01";
$VERSION = eval $VERSION;

Makefile.PL vs Build.PL

Perl modules typically utilize one of two different build systems:

• ExtUtils::MakeMaker
• Module::Build

The two different styles are easily recognizable: ExtUtils::MakeMaker employs the Makefile.PL file, and it's the 'classical' approach; Module::Build is the (relatively) new kid on the block, with support for things ExtUtils::MakeMaker cannot do. While Module::Build was designed as a long-term replacement for ExtUtils::MakeMaker, it turned out that Module::Build lacks proper upstream maintenance thus favoring Module::Build does not look like a good idea now.

There are more build systems:

ExtUtils::MakeMaker

Best practices for the latest Fedora

BuildArch:      noarch
BuildRequires:  coreutils
BuildRequires:  make
BuildRequires:  perl-generators
BuildRequires:  perl-interpreter
BuildRequires:  perl(ExtUtils::MakeMaker) >= 6.76

%build
perl Makefile.PL INSTALLDIRS=vendor NO_PACKLIST=1 NO_PERLLOCAL=1
%{make_build}

%install
%{make_install}
%{_fixperms} $RPM_BUILD_ROOT/*

%check
make test

%files
%{perl_vendorlib}/…
%{_mandir}/man3/…

# Non-noarch packages
BuildRequires:  coreutils
BuildRequires:  findutils
BuildRequires:  gcc
BuildRequires:  make
BuildRequires:  perl-devel
BuildRequires:  perl-generators
BuildRequires:  perl-interpreter
BuildRequires:  perl(ExtUtils::MakeMaker) >= 6.76

%build
perl Makefile.PL INSTALLDIRS=vendor NO_PACKLIST=1 NO_PERLLOCAL=1 OPTIMIZE="$RPM_OPT_FLAGS"
%{make_build}

%install
%{make_install}
find $RPM_BUILD_ROOT -type f -name '*.bs' -size 0 -delete
%{_fixperms} $RPM_BUILD_ROOT/*

%check
make test

%files
%{perl_vendorarch}/auto/…
%{perl_vendorarch}/…
%{_mandir}/man3/…

The %{_fixperms} macro is used for setting the write permissions of the installed files (CPAN RT#40976). The macro is implement using chmod command, hence the build dependency on coreutils.

Historical notes for old packages

• perl Makefile.PL NO_PERLLOCAL=1 – Since 6.75_01 (F21), it's possible to disable updating perllocal.pod log file by passing NO_PERLLOCAL=1 argument to Makefile.PL instead of installing with make pure_install.

• perl Makefile.PL NO_PACKLIST=1 – Since 6.75_01 (F21), it's possible to disable creating packlist files by passing NO_PACKLIST=1 argument to Makefile.PL. The packlist files clashe with RPM infrastructure and otherwise they must be deleted in %install section with find $RPM_BUILD_ROOT -type f -name .packlist -delete.

• make pure_install DESTDIR – Since 6.06_01 (all Fedoras), it's possible to use standard DESTDIR argument to make (pure_)install action instead of old PERL_INSTALL_ROOT. It points to root where files are installed to. It's recommended to apply this change to cpanspec output.

• make pure_install – make install modifies global perllocal.pod log file to list all installed Perl distributions. This duplicates and clashes with RPM infrastructure. So one has to use pure_install instead of the install to disable the perllocal.pod feature. This is implemented in cpanspec already.

• find $RPM_BUILD_ROOT -depth -type d -delete – It's not necessary to delete empty directories as all Fedoras' ExtUtils::MakeMakers do not create empty directories any more. It's recommended to remove this line from %install section of cpanspec output.

• Requires: perl(:MODULE_COMPAT_%(eval "`%{__perl} -V:version`"; echo $version)) – It's not necessary to require this RPM dependency symbol explicitly. It's automatically generated by build-required perl-generators since perl-generators-1.16 (F36).

Module::Build::Tiny

Simplified reimplementation of Module::Build. Beware it does not support destdir=foo syntax like Module::Build, one has to use --destdir=foo (CPAN RT #85006).

inc::Module::Install 

Bundled ExtUtils::MakeMaker guts. Upstream ships ExtUtils::MakeMaker modules in ./inc directory. While bundling configure-time dependencies is allowed in Fedora, one has to declare all used ./inc modules dependencies which is painful and error prone. Easier way is to prune ./inc and build-require relevant modules.

XS modules

XS modules are written in C and need a C compiler and Perl header files for building. Perl header files are provided by perl-devel package, C compiler by gcc package. You must build-require them.

If a package build script is driven by Module::Build, you will have to build-require perl(ExtUtils::CBuilder) too. Then it's possible to leave out a dependency on gcc. Otherwise you will obtain this error:

+ perl Build.PL installdirs=vendor 'optimize=-O2 -g [...]'
Can't locate ExtUtils/CBuilder.pm in @INC [...]

This ensures that it's possible to build noarch packages without having gcc or perl-devel on the system. See bug #1547165 for more details.

Tests

Tests / build steps requiring network access

This happens from time to time. Some package's tests (or other steps, e.g. signature validation) require network access to return success, but their actual execution isn't essential to the proper building of the package. In these cases, it's often nice to have a simple, transparent, clean way of enabling these steps on your local system (for, e.g., maximum testing), but to have them disabled when actually run through the buildsys/mock.

One easy way to do this is with the "--with" system of conditionals rpmbuild can handle. Running, e.g., "rpmbuild --with network_tests foo.src.rpm" is analagous to including a "--define '_with_network_tests 1'" on the command line. We can test for the existance of that conditional, and take (or not take!) certain actions based on it.

See, e.g., the perl-POE-Component-Client-HTTP spec file for an example.

One way to indicate this inside your spec is to prepend a notice along the lines of:

# some text
# about the change

Then, at the point of the operation, e.g. "make test", that needs to be disabled silently under mock to enable the package build to succeed:

%check
%{?!_with_network_tests:rm t/01* t/02* t/09* t/11* t/50* t/54*}

make test

Now to execute local builds with the network bits enabled, either call rpmbuild with "--with network_tests" or add the line "%_with_network_tests 1" to your ~/.rpmmacros file. Remember to test with _with_network_tests undefined before submitting to the buildsys, to check for syntax errors!

Tests require X11 server

Some Perl bindings to graphical toolkits deliver tests that require access to X11 server. rpmbuild was changing its opinion on DISPLAY environment variable. Currently the variable is unset when running locally or in Koji. If you want to run X11 tests, and you want it, you could do it using Xvfb X11 server implementation:

%global use_x11_tests 1

%if %{use_x11_tests}
# X11 tests:
BuildRequires:  xorg-x11-server-Xvfb
BuildRequires:  font(:lang=en)
%endif

%check
%if %{use_x11_tests}
    xvfb-run -d make test
%else
    make test
%endif

Here the X11 tests are conditionalized by use_x11_tests boolean macro. Real example can be found in perl-Padre or perl-Tk-Pod spec files.

If xvfb-run script provided with xorg-x11-server-Xvfb package became unavailable (there were attempts to remove it), you could run Xvfb manually like this:

%check
%if %{use_x11_tests}
    xinit /bin/sh -c 'rm -f ok; make test && touch ok' -- /usr/bin/Xvfb :666
    test -e ok  
%else
  make test
%endif

Note xinit returns exit code of X server, not code of X client.

Parallel tests

Tests that are executed via Test::Harness (a default for ExtUtils::MakeMaker build script, invoked using make test) can be explicitly parallelized by adding jN to HARNESS_OPTIONS environment variable, where N is a number of threads that can be obtained from %{_smp_mflags} macro value like this:

%check
export HARNESS_OPTIONS=$(perl -e \
    'for (@ARGV) { $j=$1 if m/\A-j(\d+)\z/; }; print "j$j" if $j' -- \
    %{?_smp_mflags})
make test

If some tests are not ready for parallel testing, their ordering can specified using a rules file (a default one is ./testrules.yml and ./t/testrules.yml). See TAP::Harness documentation for more details.

Enumerating perl() dependencies

It's important to specify all used Perl modules when building the package using BuildRequires and to make sure the resulting binary packages will list all Perl modules needed for running the packaged code. Because Perl is a dynamic language, the Perl modules can be loaded during a Perl code execution. Because Perl is a Turing-complete language, there is no other way how enumerate all used Perl modules other than executing the Perl code. And since execution path through the code usually depends on an input (or environment), you would have to execute the Perl code with all possible inputs to be sure you had covered all the possibilities. Since it is extremely difficult, there is no feasible way to automate the task of specifying all requires dependencies. However we have some more or less helpful approximations:

Read the code

This is a tedious method but probably the most reliable. It's especially valuable for dynamic cases like eval {use $foo; } or require Foo if $ENV{SOME_ENVIRONMENT_VARIABLE} or use Moose; with 'AMoosePlugn';.

Execute the code

Quite spectacular method to verify that a module is not used when building a package is to mangle the module name in the code. E.g. change use Foo; into use XXX::Foo and run the tests as specified in the %check section of your spec file. If the Foo is actually need, you the tests should fail with an error message that XXX::Foo Perl module could not be loaded.

Read META files

Each CPAN distribution should contain META.json or META.yml file in a root directory. Those are (usually) manually maintained list of dependencies used by CPAN clients. Thus they usually contain a minimal set of dependencies to build and run the package. However, they often miss modules bundled with upstream's perl (e.g. Socket or warnings). They also often skip dependencies for performing optional tests or code that author considers as an optional feature not needed by everyone. Also because these files are maintained manually, they often contain dependencies for an already removed code.

These files are useful for retrieving minimal required versions of Perl modules. While Perl supports use Foo 1.234; syntax to specify a minimal Foo version, many people do not write it into the code, but rather only into META files.

tangerine scanner

A Fedora package tangerine delivers a same-named tool that recursively traverses a file system, parses a Perl code and reports Perl modules that the code could attempt to load. This tool is quite comprehensive (i.e. it covers cases like use parent 'Foo'; where both parent and Foo" are correct answers). On the other hand it's quite slow with many dependencies. (That's the reason why Fedora does not use when building RPM packages.)

Perl generators

perl-generators package delivers rpmbuild plugins that are used when building RPM packages to autogenerate Perl dependencies. You pass files to scan to perl.req script on a standard input or as positional arguments. This tool is designed to be fast and to provide as few false positives as possible. Thus it does not discover all possible dependencies (e.g. indented require Foo; is suppressed by a design).

Other tools

• perl-Module-ScanDeps – a static analyzer similar to tangerine.
• perl-Perl-PrereqScanner – a static analyzer used by DistZilla.
• perl-Module-Depends – a META.yml parser.

Autogenerated dependencies

Filtering autogenerated dependencies does not work

Unfortunately rpm offers (and removes) filtering mechanism on each new release and not all supported mechanisms are compatible.

If you package for Fedora 16 and higher, use %__*_exclude macros. If you package for F15 and older, use %filter_* macros.

If you want to have unified spec file for all Fedoras, you can use both of them, but remember %__*_exclude style inhibits %filter_* style and %__*_exclude style is supported since rpm 4.9. If you use %perl_default_filter, you need to put it in between the two styles to take effect properly (see some early Fedora 16 packages for examples).

Since Fedora 16, %perl_default_filter uses %__*_exclude style, so if you use %perl_default_filter, you need to define filtering in the %__*_exclude style too.

Modules provided by private shared objects are not auto-provided by rpmbuild

Some CPAN packages contain native code and the native code can provide Perl module. E.g. XS file contains:

MODULE=Wx PACKAGE=Wx::Process

and it's compiled into /usr/lib64/perl5/vendor_perl/auto/Wx/Wx.so. When including the main package Wx, the Wx::Process becomes accessible and other module can use it:

perl -MWx -e 'use base qw(Wx::Process);'

Problem is rpmbuild discovers Requires for perl(Wx::Process), but does not discover the Provides. This leads to unresolvable dependencies in RPM repository.

Until rpmbuild will be fixed, you need to provide modules declared only in shared objects manually:

%package Wx
Provides: perl(Wx::Process)

You can use a script to do it for you:

cd Wx-*
for i in `grep -r "PACKAGE=" * | cut -d " " -f 2 | sed 's|PACKAGE=|perl(|g' | grep "Wx::" | sort -n |uniq`; do
  printf "Provides: $i)\\n";
done

This problem is described in bug #675992.

Autoprovides are missing version specifier

Not all submodules define their version. This is not good for current RPM.

Then rpmbuild generates following Provides:

$ rpm -q --provides -p perl-Statistics-Basic-1.6602-1.fc14.noarch.rpm 
perl(Statistics::Basic) = 1.6602
perl(Statistics::Basic::ComputedVector)  
perl(Statistics::Basic::Correlation)  
[…]
perl-Statistics-Basic = 1.6602-1.fc14

According Paul Howarth, unversioned Provides satisfies versioned requires. E.g. if perl-Foo requires perl(Statistics::Basic::ComputedVector) >= 2.000 it will be satisfied by perl-Statistics-Basic-1.6602 because it provides perl(Statistics::Basic::ComputedVector).

IMHO, it sounds like a bug in RPM, you can patch it by following Provides filtering:

%filter_from_provides s/^\(perl(Statistics::Basic\>[^=]*\)$/\1 = %{version}/

It will add the version specifier equaled to package version to each sub-module missing any version specifier.

Bugs in RPM dependency generator

If you find a bug or shot-coming in dependency generator, report it into Bugzilla for rpm component, make proper subject (e.g. write prefix Perl dependency generator:), and make the bug blocking bug 694496 ((requirements, rpm) rpm requirements - provides/requires (tracker)).

rpmlint errors

file-not-utf8

Problem

W: perl-Class-MakeMethods file-not-utf8 /usr/share/man/man3/Class::MakeMethods::Docs::ReadMe.3pm.gz
W: perl-Class-MakeMethods file-not-utf8 /usr/share/man/man3/Class::MakeMethods::Attribute.3pm.gz

Solution

Convert the errant file to UTF-8. Assuming the codepage the file is currently under is ISO-8859-1, this will do the trick (often by reviewers wanted in %prep section, in %build for generated man pages):

cd blib/man3
for i in Docs::ReadMe.3pm Attribute.3pm ; do
  iconv --from=ISO-8859-1 --to=UTF-8 Class::MakeMethods::$i > new
  mv new Class::MakeMethods::$i
done

If you are using iconv, you should be BR'ing it, but it's in glibc-common, which is installed anyway...

private-shared-object-provides

Problem

W: private-shared-object-provides /usr/lib64/perl5/vendor_perl/auto/File/Map/Map.so Map.so()(64bit)

The Map.so is a private shared library dynamically loaded by XS loader when a Perl binding to a C library is called. These files are not intended for public use and they must be filtered from Provides. In addition, the files have name similar to original C libraries which can clashes while resolving RPM dependencies while installing packages.

Solution

Filter the unneeded Provides by  %{?perl_default_filter} macro.

script-without-shellbang

Problem

Rpmlint returns something to the effect of:

E: perl-WWW-Myspace script-without-shellbang /usr/lib/perl5/vendor_perl/5.8.8/WWW/Myspace/Comment.pm
E: perl-WWW-Myspace script-without-shellbang /usr/lib/perl5/vendor_perl/5.8.8/WWW/Myspace/MyBase.pm
E: perl-WWW-Myspace script-without-shellbang /usr/lib/perl5/vendor_perl/5.8.8/WWW/Myspace/FriendAdder.pm

Solution

This error is caused by the exec bit being set on one or more .pm files. The solution is to strip the exec bit, for example, in the %install section:

find %{buildroot} -type f -name '*.pm' -exec chmod -x {} 2>/dev/null ';'

wrong-script-interpreter

Problem

E: wrong-script-interpreter /usr/share/perl5/vendor_perl/ExtUtils/xsubpp perl

Solution

Replace incorrect shebang at the end of %install section:

sed -i 's|#!perl|#!/usr/bin/perl|' %{buildroot}%{perl_vendorlib}/ExtUtils/xsubpp

Compatibility issues after upgrading Perl

ExtUtils::MakeMaker overrides CCFLAGS with Perl 5.14

Problem

When compiling package driven by ExtUtils::MakeMaker that utilizes CCFLAGS argument, you get message:

Not a CODE reference at /usr/lib/perl5/DynaLoader.pm

Solution

This is bug in ExtUtils::MakeMaker probably. It replaces CCFLAGS instead of adding them to Perl default ccflags. See message perl 5.14 ExtUtils::MakeMaker overriding CCFLAGS in perl-devel mailing list and Debian bug report.

New Perl specific spec file macros

If you find out that some code snippets repeat in your spec files, you could say: Hey, there should be a macro for that! Then propose the macro for inclusion into /etc/rpm/macros.perl. The file is owned by perl package and is automatically included by rpmbuild tool. Ask perl package maintainer for adding the macro.