View source for Features/FullyTestablePerl

{{admon/important | Incomplete! | This page is incomplete, and liable to change significantly before being proposed!}}

<!-- All fields on this form are required to be accepted by FESCo.
 We also request that you maintain the same order of sections so that all of the feature pages are uniform.  -->

<!-- The actual name of your feature page should look something like: Features/YourFeatureName.  This keeps all features in the same namespace -->

= Feature Name =
Fully Testable Perl.

== Summary ==
<!-- A or sentence summarizing what this feature is and what it will do.  This information is used for the overall feature summary page for each release. -->

Provide the ability to re-test installed Perl packages (primarily [[http://www.cpan.org|CPAN]]) by automatically providing the test suites of each distribution as their own "perl-Foo-tests" subpackage, and providing a framework to test.

== Owner ==
<!--This should like to your home wiki page.  Your home wiki page should include an email address that you can be reached at should people want to contact you about helping with your feature or to request status -->

* Name: [[User:Cweyl| Chris Weyl]]

== Current status ==

* Targeted release: [[Releases/{{FedoraVersion||next}} | {{FedoraVersion|long|next}} ]] 
* Last updated: (DATE)
* Percentage of completion: XX%

{{Progress_bar}}

<!-- CHANGE THE "FedoraVersion" TEMPLATES ABOVE TO PLAIN NUMBERS WHEN YOU COMPLETE YOUR PAGE. -->

== Detailed Description ==
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->

Perl has a long history and culture of testing, which has resulted in a very high percentage of the packages on the CPAN (Comprehensive Perl Archive Network) containing significant test suites.  While these test suites are executed at build time, a large number of modern Perl distributions (e.g. Moose, DBIx::Class, Catalyst, etc) depend on a significant number of other dists, which may be owned by a different maintainer and updated independently.

That is, the tests results as executed at build are only valid so long as its environment (the other Perl dists the particular package may depend on) is not changed from the environment this package is built with.  Additionally, there are often tests which are available but disabled during %make check as they may cause a circular build dep or require resources not available under the buildsystem (network or $DISPLAY access, or test databases, etc).

This feature proposes to automatically bundle the dists test suites into a -tests subpackage without requiring additional maintainer work (a la debuginfo), which can then be installed to provide the capability to retest functionality post installation.

== Benefit to Fedora ==
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new feature, what capabilities does it bring? Why will Fedora become a better distribution or project because of this feature?-->

Being able to execute the tests of a Perl dist post-build will leverage existing code (the test suites) to allow the end user to perform sanity checks and rule out problems with the underlying code when tracking down bugs.  

The net benefit to Fedora is to create an environment where end users (support, programmers, etc) will be able to execute the test suites of the 900+ CPAN dists Fedora packages natively, to track down bugs, test functionality the buildsystem was unable to provide, or just validate their environment.

=== Example ===

e.g. Let's say someone is using Catalyst (perl-Catalyst-Runtime) to run a website, and is in particular using Catalyst::Authentication::Store::DBIx::Class (perl-Catalyst-Plugin-Authentication-Store-DBIx-Class).  A new release of DBIx::Class (perl-DBIx-Class) was just built and released, which introduced a subtle bug causing errors in Catalyst::Authentication::Store::DBIx::Class.  While this is a contrived example, the value of being able to easily pull and execute the Catalyst::Authentication::Store::DBIx::Class test suite to "re-check" against the new environment (that is, the newer DBIx::Class than was available at build time)

== Scope ==
<!-- What work do the developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->

* work with the perl-qa mailing list for input on how to retain these tests, and potentially other issues which could be made "standard"
* rpm-build macros:  overload the %debug_package macro to automatically create -tests subpackages for Perl dists.
** Working proof-of-concept rpm macros overloading %debug_package can be viewed here: [[Features/FullyTestablePerl/RpmMacros]]
* Create a test-framework: e.g. to test perl-Moose and all its deps, an end user could execute "perl-tests Moose" and step back.


== Test Plan ==
<!-- This does not need to be a full-fledged document.  Describe the dimensions of tests that this feature is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The QA team will turn this information into a more complete test plan.  The more specific you can be, the better the final test plan will be. 

A good Test Plan should answer these four questions:

0. What special hardware / data / etc. is needed (if any)?
1. How do you prepare your system to test this feature? What packages
need to be installed, config files edited, etc.?
2. What specific actions do you perform to check that the feature is
working like it's supposed to?
3. What are the expected results of those actions?

-->

== User Experience ==
<!-- If this feature is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->

== Dependencies ==
<!-- What other packages (RPMs) depend on this package?  Are there changes outside the developers' control on which completion of this feature depends?  In other words, completion of another feature owned by someone else and might cause you to not be able to finish on time or that you would need to coordinate?  Other upstream projects like the kernel (if this is not a kernel feature)? -->

== Contingency Plan ==
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "None necessary, revert to previous release behaviour."  Or it might not.  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->

== Documentation ==
<!-- Is there upstream documentation on this feature, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->


== Release Notes ==

<!-- The Fedora Release Notes inform end-users about what is new in the release.  Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ -->
<!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns.  If there are any such changes involved in this feature, indicate them here.  You can also link to upstream documentation if it satisfies this need.  This information forms the basis of the release notes edited by the documentation team and shipped with the release. -->

== Comments and Discussion ==

* See [[Talk:Features/FullyTestablePerl]]  <!-- This adds a link to the "discussion" tab associated with your page.  This provides the ability to have ongoing comments or conversation without bogging down the main feature page -->


----
[[Category:Perl]]
[[Category:FeaturePageIncomplete]]
<!-- When your feature page is completed and ready for review -->
<!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler -->
<!-- After review, the feature wrangler will move your page to Category:FeatureReadyForFesco... if it still needs more work it will move back to Category:FeaturePageIncomplete-->
<!-- A pretty picture of the page category usage is at: https://fedoraproject.org/wiki/Features/Policy/Process -->