From Fedora Project Wiki
No edit summary
No edit summary
Line 12: Line 12:
<!-- A or sentence summarizing what this feature is and what it will do.  This information is used for the overall feature summary page for each release. -->
<!-- A or sentence summarizing what this feature is and what it will do.  This information is used for the overall feature summary page for each release. -->


Provide the ability to re-test installed Perl packages (primarily [[http://www.cpan.org|CPAN]]) by automatically providing the test suites of each distribution as their own "perl-Foo-tests" subpackage, and providing a framework to test.
Provide the ability to re-test installed [[Perl]] packages (primarily [[http://www.cpan.org|CPAN]]) by automatically providing the test suites of each distribution as their own "perl-Foo-tests" subpackage, and providing a framework to test.


== Owner ==
== Owner ==
Line 32: Line 32:
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->


Perl has a long history and culture of testing, which has resulted in a very high percentage of the packages on the CPAN (Comprehensive Perl Archive Network) containing significant test suites.  While these test suites are executed at build time, a large number of modern Perl distributions (e.g. Moose, DBIx::Class, Catalyst, etc) depend on a significant number of other dists, which may be owned by a different maintainer and updated independently.
Perl has a long history and culture of testing, which has resulted in a very high percentage of the packages on the CPAN (Comprehensive Perl Archive Network) containing significant test suites.  While these test suites are executed at build time, a large number of modern Perl distributions (e.g. {{CPAN|Moose}}, {{CPAN|DBIx::Class}}, [[Catalyst]], etc) depend on a significant number of other dists, which may be owned by a different maintainer and updated independently.


That is, the tests results as executed at build are only valid so long as its environment (the other Perl dists the particular package may depend on) is not changed from the environment this package is built with.  Additionally, there are often tests which are available but disabled during %make check as they may cause a circular build dep or require resources not available under the buildsystem (network or $DISPLAY access, or test databases, etc).
That is, the tests results as executed at build are only valid so long as its environment (the other Perl dists the particular package may depend on) is not changed from the environment this package is built with.  Additionally, there are often tests which are available but disabled during %make check as they may cause a circular build dep or require resources not available under the buildsystem (network or $DISPLAY access, or test databases, etc).
Line 44: Line 44:


The net benefit to Fedora is to create an environment where end users (support, programmers, etc) will be able to execute the test suites of the 900+ CPAN dists Fedora packages natively, to track down bugs, test functionality the buildsystem was unable to provide, or just validate their environment.
The net benefit to Fedora is to create an environment where end users (support, programmers, etc) will be able to execute the test suites of the 900+ CPAN dists Fedora packages natively, to track down bugs, test functionality the buildsystem was unable to provide, or just validate their environment.
Having the test suite used to build the package under [[koji]] would allow someone to test their modules without having to move away from the Fedora perl packages.  It will:
* Make finding regressions between multiple packages owned by multiple maintainers easier.
* Provide the end user with the tools to test, debug and validate their environments with the same test suites originally used to build the modules.
* Leverage existing RPM functionality to create the -tests subpackages without needing any package maintainer interaction, just as -debuginfo packages are created today.
* Provide an additional source of documentation (e.g. {{CPAN|Moose::Role}} explicitly refers one to the test suite as the "best documentation").


=== Example ===
=== Example ===


e.g. Let's say someone is using Catalyst (perl-Catalyst-Runtime) to run a website, and is in particular using Catalyst::Authentication::Store::DBIx::Class (perl-Catalyst-Plugin-Authentication-Store-DBIx-Class).  A new release of DBIx::Class (perl-DBIx-Class) was just built and released, which introduced a subtle bug causing errors in Catalyst::Authentication::Store::DBIx::Class.  While this is a contrived example, the value of being able to easily pull and execute the Catalyst::Authentication::Store::DBIx::Class test suite to "re-check" against the new environment (that is, the newer DBIx::Class than was available at build time)
For example, {{CPAN|Moose}} is a modern metaobject-based class framework for Perl 5. It's been around for awhile, has a stable user API, and is extremely powerful.  It has a slew of dependencies, as well as extensions in the [http://cpan.uwinnipeg.ca/dist/MooseX/ MooseX::*] namespace.  Sometimes these extensions take advantage of the metaobject system -- those that do can be highly sensitive to backend changes in both Moose and {{CPAN|Class::MOP}}.  {{CPAN|MooseX::AttributeHelpers}} is one such extension -- providing alternate attribute metaclass objects for different data structure.  Indeed, Moose's build script explicitly checks for certain levels of modules it knows it will break.


== Scope ==
== Scope ==

Revision as of 22:39, 8 November 2008

Incomplete!
This page is incomplete, and liable to change significantly before being proposed!


Feature Name

Fully Testable Perl.

Summary

Provide the ability to re-test installed Perl packages (primarily [[1]]) by automatically providing the test suites of each distribution as their own "perl-Foo-tests" subpackage, and providing a framework to test.

Owner

Current status

  • Targeted release: Fedora 42
  • Last updated: (DATE)
  • Percentage of completion: XX%


Detailed Description

Perl has a long history and culture of testing, which has resulted in a very high percentage of the packages on the CPAN (Comprehensive Perl Archive Network) containing significant test suites. While these test suites are executed at build time, a large number of modern Perl distributions (e.g. Moose, DBIx::Class, Catalyst, etc) depend on a significant number of other dists, which may be owned by a different maintainer and updated independently.

That is, the tests results as executed at build are only valid so long as its environment (the other Perl dists the particular package may depend on) is not changed from the environment this package is built with. Additionally, there are often tests which are available but disabled during %make check as they may cause a circular build dep or require resources not available under the buildsystem (network or $DISPLAY access, or test databases, etc).

This feature proposes to automatically bundle the dists test suites into a -tests subpackage without requiring additional maintainer work (a la debuginfo), which can then be installed to provide the capability to retest functionality post installation.

Benefit to Fedora

Being able to execute the tests of a Perl dist post-build will leverage existing code (the test suites) to allow the end user to perform sanity checks and rule out problems with the underlying code when tracking down bugs.

The net benefit to Fedora is to create an environment where end users (support, programmers, etc) will be able to execute the test suites of the 900+ CPAN dists Fedora packages natively, to track down bugs, test functionality the buildsystem was unable to provide, or just validate their environment.

Having the test suite used to build the package under koji would allow someone to test their modules without having to move away from the Fedora perl packages. It will:

  • Make finding regressions between multiple packages owned by multiple maintainers easier.
  • Provide the end user with the tools to test, debug and validate their environments with the same test suites originally used to build the modules.
  • Leverage existing RPM functionality to create the -tests subpackages without needing any package maintainer interaction, just as -debuginfo packages are created today.
  • Provide an additional source of documentation (e.g. Moose::Role explicitly refers one to the test suite as the "best documentation").

Example

For example, Moose is a modern metaobject-based class framework for Perl 5. It's been around for awhile, has a stable user API, and is extremely powerful. It has a slew of dependencies, as well as extensions in the MooseX::* namespace. Sometimes these extensions take advantage of the metaobject system -- those that do can be highly sensitive to backend changes in both Moose and Class::MOP. MooseX::AttributeHelpers is one such extension -- providing alternate attribute metaclass objects for different data structure. Indeed, Moose's build script explicitly checks for certain levels of modules it knows it will break.

Scope

  • work with the perl-qa mailing list for input on how to retain these tests, and potentially other issues which could be made "standard"
  • rpm-build macros: overload the %debug_package macro to automatically create -tests subpackages for Perl dists.
  • Create a test-framework: e.g. to test perl-Moose and all its deps, an end user could execute "perl-tests Moose" and step back.


Test Plan

User Experience

Dependencies

Contingency Plan

Documentation

Release Notes

Comments and Discussion


Retrieved from "https://fedoraproject.org/w/index.php?title=Features/FullyTestablePerl&oldid=58684"