UpdatePolicy(draft)

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

There are currently several competing proposals for what an update policy should look like. Listing them all on this page.

mjg's proposal

Introduction

We assume the following axioms:

1) Updates to stable that result in any reduction of functionality to the user are unacceptable.

2) It is impossible to ensure that functionality will not be reduced without sufficient testing.

3) Sufficient testing of software inherently requires manual intervention by more than one individual.

Proposal

The ability for maintainers to flag an update directly into the updates repository will be disabled. Before being added to updates, the package must receive a net karma of +3 in Bodhi.

It should be noted that this does not require that packages pass through updates-testing. The package will appear in Bodhi as soon as the update is available. If sufficient karma is added before a push occurs, and the update is flagged for automatic pushing when the karma threshold is reached, the update will be pushed directly to updates.

It is the expectation of Fesco that the majority of updates should easily be able to garner the necessary karma in a minimal space of time. Updates in response to functional regressions should be coordinated with those who have observed these regressions in order to confirm that the update fixes them correctly.

At present, this policy will not apply to updates that are flagged as security updates.

The future

Defining the purpose of Fedora updates is outside the scope of Fesco. However, we note that updates intended to add new functionality are more likely to result in user-visible regressions, and updates that alter ABI or API are likely to break local customisations even if all Fedora packages are updated to match. We encourage the development of mechanisms that ensure that users who wish to obtain the latest version of software remain able to do so, while not tending to introduce functional, UI or interface bugs for users who wish to be able to maintain a stable platform.

mether's proposal

Requirements for Critpath packages

• Must go through updates-testing repository
• Only major bug fixes and security fixes.
• Must go through updates testing repository even for security fixes

Rationale: Expedited security fixes have caused some serious regressions in the past (D-Bus, Bind, Thunderbird updates etc).

• Requires QA team to sign off on these updates and I will leave them

to define the criteria for it. I believe the criteria should be based on feedback from testers rather than the number of days.

• Exceptions or expedited update requests must go via release engineering

Critical_Path_Packages critical path packages list

Requirements for Non-critical path packages

• Don't blindly push every upstream release as update
• Preserve stability and avoid unexpected changes and push updates with

enhancements only if the benefit is considered worth the risk of potential regressions

Recommendations

• Run AutoQA on all updates
• Hookup PackageKit to updates-testing repo and allow users to opt-in

and provide feedback easily

• Evaluate extending the criteria based on how well we succeed with a

more conservative update policy for critical path packages

notting's Proposal

Introduction

We assume the following axioms:

1) Updates to stable that result in any reduction of functionality to the user are unacceptable.

2) It is impossible to ensure that functionality will not be reduced without sufficient testing.

3) Sufficient testing of software inherently requires manual intervention by more than one individual.

Proposal

For a package to be pushed to the stable updates repository, it must meet the following criteria.

1. All updates (even security) must pass AutoQA tests. Rationale: If a package breaks dependencies, does not install, or fails other obvious tests, it should not be pushed. Period. Obviously, this proposal would not be enacted until AutoQA is live.
2. Updates that constitute a part of the 'important' package set (defined below) must follow the rules as defined for critical path packages for pending releases, meaning that they require positive karma from releng and/or QA before they go stable. This also includes security updates for these packages. The 'important' package set is defined as the following:
   • The current critical path package set
   • All major desktop environments' core functionality (GNOME, KDE, XFCE, LXDE)
   • Package updating frameworks (gnome-packagekit, kpackagekit)
   • Major desktop productivity apps. An initial list would be firefox, kdebase (konqueror), thunderbird, evolution, kdepim (kmail).

   Rationale: These are the sets of packages where regressions most affect users, and would most prevent them from Getting Their Work Done. Furthermore, while I can accept that there may be some packages in Fedora that cannot find a significant enough testing base for all potential updates, I reject the notion that any desktop widely used enough that we deploy a image or spin for it would fit into that category. I accept that this places a larger burden on QA, and would expect them to be able to contribute testing to this initiative.

3. All other non-security updates must either:
   1. reach their specified bodhi karma threshold
   2. spend some minimum amount of time in updates-testing, with a tracked number of downloads.

   Proposed time would be one week, but is open to negotiation.

   Rationale: We do want additional eyes on updates wherever possible. We do have one Fedora mirror that Fedora infrastructure controls; we should be able to mine this server for data on updates-testing downloads.

Any update that wants to bypass these procedures would need majority approval from FESCo.

kparal's proposal

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Introduction

The purpose of this document is to present a policy/workflow that will be used when handling package updates. It specifies when, where and which kind of package updates may occur.

Scope

• This policy concerns only currently supported Fedora releases.
  • Rules for the current pending release will be added to this document. Rules for Rawhide may come in the future.
• Security updates that fall into the Important package set are subject to this policy. Security updates that do not fall into this category are not subject to this policy.

Workflows

Two workflows are defined, for different sets of packages. The sets are important packages and other packages.

Important package set

The important package set consists of:

• The current critical path package set
• All major desktop environments' core functionality (GNOME, KDE, XFCE, LXDE)
• Package updating frameworks (gnome-packagekit, kpackagekit)
• Major desktop productivity apps. An initial list would be firefox, kdebase (konqueror), thunderbird, evolution, kdepim (kmail).

Other package set

The other package set consists of all packages not in the important package set.

Important workflow

All package updates are submitted to the 'updates-testing' repository. Then they await initial sanity test acceptance, a process owned by the Quality Assurance team. Acceptance is granted according to Initial requirements. The decision is based on the result of Package update acceptance test plan, which is executed automatically by AutoQA. When the approval is granted, the update is pushed to the 'updates-testing' repository.

Next, the updates must meet all of these requirements:

• Updates must reach a specified positive Bodhi karma threshold provided by a defined team (e.g. proventesters). These qualified testers should consider feedback (especially negative feedback) from other testers in deciding whether to provide approval, as well as their own tests.
• Updates must spend some minimum amount of time in updates-testing. This requirement does not hold for security updates.

After submitting to 'updates' repository another round of automated AutoQA acceptance tests is run. Acceptance is granted according to Stable requirements. As a final point Release Engineering ensures that Stable requirements were satisfied and they sign off and push the update to 'updates'.

Other workflow

All package updates are submitted to the 'updates-testing' repository. Then they await initial sanity test acceptance, a process owned by the Quality Assurance team. Acceptance is granted according to Initial requirements. The decision is based on the result of Package update acceptance test plan, which is executed automatically by AutoQA. When the approval is granted, the update is pushed to the 'updates-testing' repository.

Next, the updates must meet at least one of these requirements:

• Updates must reach a specified positive Bodhi karma threshold . Anyone may provide karma to an update.
• Updates must spend some minimum amount of time in updates-testing.

Proposed time would be one week, but is open to negotiation. We can track downloads with our one Fedora-infrastructure controlled mirror as mechanism to see what usage the package is getting.

When at least one of the constraints is satisfied the package maintainer may submit the update to the 'updates' repository.

After submitting to 'updates' repository another round of automated AutoQA acceptance tests is run. Acceptance is granted according to Stable requirements. As a final point Release Engineering ensures that Stable requirements were satisfied and they sign off and push the update to 'updates'.

Initial Requirements

• AutoQA: Test Package Sanity - Packages must be able to be installed, removed, upgraded, etc.
• AutoQA: Packages must not break dependencies in updates + updates-testing.
• AutoQA: Packages must not introduce file/package conflicts in updates + updates-testing.
• + maybe some more

Stable Requirements

• AutoQA: Initial requirements are re-checked and met.
• AutoQA: Packages must not break upgrade path.
• RelEng: The package update type falls into the list of allowed update types (document not approved yet, just a sample).
• + maybe some more

Exception Process

Package maintainers may ask for exception, when:

• they don't agree with QA rejecting their update
• they need to push an update without fully adhering to policy requirements
  • this is suitable for high-important bug-fix updates, which repair severe regressions/breakages
  • some other candidates?
• they don't agree with RelEng rejecting their update

The process of asking for an exception is yet to be defined. It can be a majority vote from FESCo.

Responsibilities

• When a serious problem is found in a not yet released updated package, package update builder is responsible for removing this update from 'updates-testing' repository. Fedora Update System must provide this option for package update builder, package maintainer, Quality Assurance team and Release Engineering team.

• The Fedora Update System is responsible for sending notifications to package update builder and package maintainer when:
  • the update receives a result from Package update acceptance test plan
  • the update has fulfilled requirements for submitting into 'updates' repository
  • the update has been accepted or rejected to be pushed to 'updates' repository by Release Engineering team
  • the update becomes available in the 'updates' repository

• Even though some security updates do not fall within this policy, the Quality Assurance team is still obliged to test the packages, after they're pushed if necessary, and report any regressions that need to be fixed to the maintainer.

References

Related links

• UpdatePolicy(draft)
• Stable Release Updates Proposal
• Package update guidelines
• User:Kparal/Package update approaches
• Release Lifecycle(draft)

adamw's hybrid

adamw wanted to show how combining notting and kparal's proposals would look. This is basically the combination of those two.

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Use discussion
Don't forget to check discussion page.

This is just a proposal
Please bear in mind that this is just a draft meant to provoke discussion. Nothing is rock-solid here and it is not an attempt of the QA team to Fedora project domination. This is a combination of the QA team's proposal and Bill Nottingham's proposal.

Introduction

The purpose of this document is to present a policy/workflow that will be used when handling package updates. It specifies when, where and which kind of package updates may occur.

Scope

• This policy concerns only currently supported Fedora releases.
  • Rules for the current pending release will be added to this document. Rules for Rawhide may come in the future.
• Security updates need not to follow this policy as the Security Response Team can bypass it to have the updates released as soon as possible.

Workflows

Two workflows are defined, for different sets of packages. The sets are important packages and other packages.

Important package set

The important package set consists of:

• The current critical path package set
• All major desktop environments' core functionality (GNOME, KDE, XFCE, LXDE)
• Package updating frameworks (gnome-packagekit, kpackagekit)
• Major desktop productivity apps. An initial list would be firefox, kdebase (konqueror), thunderbird, evolution, kdepim (kmail).

Other package set

The other package set consists of all packages not in the important package set.

Important workflow

All package updates are submitted to the 'updates-testing' repository. Then they await initial sanity test acceptance, a process owned by the QQuality Assurance team. FIXME: this is the filter point for really basic automated sanity tests - dependency sanity, major packaging errors. Acceptance is granted according to Requirements. The decision is based on the result of Package update acceptance test plan, which is executed automatically by AutoQA. When the approval is granted, the update is pushed to the 'updates-testing' repository.

Next, the package updates must be tested and approved by the QA or Release Engineering teams. Approval consists of a +1 vote in Bodhi. Qualified testers should consider feedback from other testers in deciding whether to provide approval, especially negative feedback, as well as their own tests. Only after that the package maintainer may push the update to the 'updates' repository.

Other workflow

Maintainers may choose to submit updates to the 'updates-testing' or directly to the 'updates' repository. Whichever repository is chosen, the update must pass the same initial sanity tests as in the Important workflow. If an update sent to 'updates-testing' has passed the sanity tests, the maintainer may push it to the 'updates' repository at any time. Maintainers are encouraged, but not required, to request additional review from the QA group or other testers and to consider the feedback in choosing whether to push the update.

Requirements

• TODO
  • Here will be the list of requirements the package update must pass. Different teams (RelEng, QA) will probably add their own specific suggestions.
• The result of Package update acceptance test plan must be ACCEPTED. It may be NEEDS_REVIEW initially and then changed to ACCEPTED by waiving errors, that is allowed.

Exception process

Package maintainers may ask for exception, when:

• they don't agree with QA rejecting their update
• they need to shorten time required for an update to spend in 'updates-testing'
  • this is allowed for high-important bug-fix updates, which repair severe regressions/breakages
  • some other candidates?
• they don't agree with RelEng rejecting their update

The process of asking for an exception is yet to be defined.

Responsibilities

• When a serious problem is found in a not yet released updated package, package update builder is responsible for removing this update from 'updates-testing' repository. Fedora Update System must provide this option for package update builder, package maintainer, Quality Assurance team and Release Engineering team.

• The Fedora Update System is responsible for sending notifications to package update builder and package maintainer when:
  • the update receives a result from Package update acceptance test plan
  • approval is granted or refused by Release Engineering team (both when submitting to 'updates-testing' or 'updates' repository)
  • the update has fulfilled requirements for submitting into 'updates' repository
  • the update becomes available in the 'updates' repository

• Even though security updates do not fall within this policy, the Quality Assurance team is still obliged to test the packages, after they're pushed if necessary, and report any regressions that need to be fixed to the maintainer.

References

Related links

• Package update guidelines
• User:Kparal/Package update approaches
• Release Lifecycle(draft)