From Fedora Project Wiki
m (→‎Identified needs: Reorder the columns to make the *need* more visible.)
No edit summary
 
(34 intermediate revisions by 5 users not shown)
Line 1: Line 1:
{{old}}
{{draft|This document will eventually contain a roadmap for implementing the Playground repository.  It will outline what the playground repository is and what groups, manpower, and other resources will be needed in order to implement it.}}
{{draft|This document will eventually contain a roadmap for implementing the Playground repository.  It will outline what the playground repository is and what groups, manpower, and other resources will be needed in order to implement it.}}


Line 14: Line 16:
=== How the repository works ===
=== How the repository works ===


Packages for the repository are built in [http://copr.fedoraproject.org/ COPR]. The COPR owner can mark the repository as a whole as being part of the Playground repository. Packages successfully built for marked COPRs are copied into the Playground Repository.
Packages for the repository are built in [http://copr.fedoraproject.org/ COPR]. The COPR owner can propose the repository as a whole for inclusion into the Playground repository by marking it as such in COPR. Repositories/packages successfully built and satisfying the Playground repository's [[#Policies|Policies]] are copied into the Playgroud repository. The one Playground repository includes many Copr repositories.
[marcela] Who is COPR owner? The project owner on COPR? We need additional feature in COPR for "mark as worth of Playground".


* How do the updates work?
* How do the updates work?
**The Playground repository follows the rolling release model. One yum/dnf repository is provided for each Fedora release-arch combination. The repository's repodata is continuously regenerated. All the builds in the COPR repositories that are selected to feed the Playground repository are composed once a day and pushed to the Playground repository and its mirrors.
**The Playground repository follows the rolling release model. Data in Playground are more repository of repositories. All the builds in the COPR repositories that are selected to feed the Playground repository are marked as Playground repository content.
**This is similar to the [[Rawhide]] repository.
**This is similar to the [[Rawhide]] repository.
**Initially, the [[Bodhi]] update system will not be used.
**Initially, the [[Bodhi]] update system will not be used.
**These decisions were made on the [http://meetbot.fedoraproject.org/fedora-meeting/2014-03-04/env-and-stacks.2014-03-04-13.00.log.html March 4, 2014 meeting].
**These decisions were made on the [http://meetbot.fedoraproject.org/fedora-meeting/2014-03-04/env-and-stacks.2014-03-04-13.00.log.html March 4, 2014 meeting], but after working with dnf plugin was changed.


* Does it have an additional testing repository?
* Does it have an additional testing repository?
**Initially, there won't be an additional testing repository. If packagers want to provide some testing packages, they can create an additional COPR that will contain these testing packages.
**Initially, there won't be an additional testing repository. If packagers want to provide some testing packages, they can create an additional COPR that will contain these testing packages.
**This decision was made on the [http://meetbot.fedoraproject.org/fedora-meeting/2014-03-04/env-and-stacks.2014-03-04-13.00.log.html March 4, 2014 meeting].
**This decision was made on the [http://meetbot.fedoraproject.org/fedora-meeting/2014-03-04/env-and-stacks.2014-03-04-13.00.log.html March 4, 2014 meeting].
* Self hosting: All packages in the Playground repository are buildable using only packages in the Fedora' main repository and the Playground repository.
** This decision was made on the [http://meetbot.fedoraproject.org/teams/env-and-stacks/env-and-stacks.2014-03-11-16.04.log.html March 11, 2014 meeting]
* Upgrades and broken dependencies
** [[FedUp]] will support upgrades with the packages in the Playground repository as if it was a generic third party repository. No special handling of the Playground repository will be implemented.
** Packages in the Playground repository may have broken dependencies.  We'll encourage maintainers to keep the package set free of these problems but by its nature (rolling, experimental, having packages who's dependencies change frequently) the Playground repository may have these problems more frequently than the Fedora's main repository.
*** There can be co-maintainers or provenpackagers [[Provenpackager_policy|Provenpackagers]] [http://meetbot.fedoraproject.org/fedora-meeting/2014-04-01/env-and-stacks.2014-04-01-12.02.log.html April 1, 2014]
** We'll also implement a way to automatically notify the package owners about broken dependencies.
** These decisions were made on the [http://meetbot.fedoraproject.org/teams/env-and-stacks/env-and-stacks.2014-03-11-16.04.log.html March 11, 2014 meeting]
* At the moment, the Playground repository is not multilib.  This may change in the future if someone devotes the time to make it happen.
** This decision was made on the [http://meetbot.fedoraproject.org/teams/env-and-stacks/env-and-stacks.2014-03-11-16.04.log.html March 11, 2014 meeting]
* Details about content:
** Signing - obs-signd for Copr packages will be used [http://meetbot.fedoraproject.org/teams/env-and-stacks/env-and-stacks.2014-04-01-12.02.log.html April 1, 2014 meeting]
** Don't distinguish packages. It's same as rpmfusion packages [http://meetbot.fedoraproject.org/fedora-meeting/2014-04-01/env-and-stacks.2014-04-01-12.02.log.html April 1, 2014]
* Reviews
** We will provide a kind of "non-blocking review service" and later we will talk about whether we should make it mandatory or not. Coprs won't be blocked by reviews results in the beginning. [http://meetbot.fedoraproject.org/fedora-meeting/2014-04-01/env-and-stacks.2014-04-01-12.02.log.html April 1, 2014]


== Identified needs ==
== Identified needs ==
Line 32: Line 53:
|| Need || How necessary || Groups to Coordinate with
|| Need || How necessary || Groups to Coordinate with
|+
|+
|| Disk space for the yum repositories (Open question -- is this mirrored?) || Necessary || Infra
|| Disk space for the yum/dnf repositories || Necessary || Infra
|-
|| Ability to mark an individual COPR for inclusion in the Playground repository || Necessary || Copr devs
|-
|| Continuously regenerating repodata || Necessary || Infra
|-
|| Daily composes of the Playground repository || Necessary || Infra
|-
|-
|| Copr deployment that's considered reliable enough to build packages for this repo || Very nice to have || Infra/Copr devs
|| Copr deployment that's considered reliable enough to build packages for this repo || Very nice to have || Infra/Copr devs
|-
|-
|| Ability to mark an individual COPR for inclusion in the Playground repository || Necessary || Copr devs
|| Build from a git repository URL and revision hash || Optional but nice to have || Copr devs
|-
|| deltarpms || Optional but nice to have || Releng
|-
|| multilib || Optional but nice to have || Releng
|-
|| Mirroring/mirrormanager || Optional || Infrastructure/releng
|-
|| Signing by obs-signd || Necessary || Copr devels
|-
|-
|| Build from a git repository URL and revision hash || Optional but nice to have || Copr devs
|| Reviews - automatic service for reviews || Optional || pingou&sochotni
|-
|-
|}
|}
Line 44: Line 79:
== Open Questions ==
== Open Questions ==


We'll need to answer these questions and by their answers, flesh out the [#Description] and add additional work items to the [#Identified_needs] section.
We'll need to answer these questions and by their answers, flesh out the [[#Description|Description]] and add additional work items to the [[#Identified_needs|Identified needs]] section.


* deltarpms?
* Taskotron - basic sanity for inclusion into Playground?


* signing?
=== Open Questions might be solved by multiple repos proposal ===
** it takes 4 months to implement in Copr
* Do we allow conflicts with packages in the main repo?


* does it need adding to mirrormanager?
* Do we allow replacement of packages in the main repo?
** Do we allow "backdoor replacement" of packages in the main repo?  ie: Let's say I have a package in the playground repo: NetworkManager2.1.  And that conflicts with NetworkManager.  Is that allowed?  Is it allowed as long as it doesn't have any virtual provides/obsoletes that would automatically allow it to replace the package in the main repo?


* will fedup support upgrades with packages there?  
* Do we allow conflicts between packages in the Playground Repo?


* Does it need to be mashed in order to get multilib support?
* Do we allow replacement of other packages in the Playground Repository?  (How do we stop this in our implementation?)
** Do we allow "backdoor replacement" in the playground repo?


* self hosting (all packages needed to build the packages are in the repo)?
* How do we deal with multiple versions of same package provided by different COPRs?
 
* Will there be any review of COPRs/packages that will become part of the Playground repository?
** We are aiming at an automatic review that will check if the COPR/packages satisfy the Playgroud repository's [[#Policies|Policies]].
** The automatic reviewing service will first attempt to do a fully automatic package review, falling back to human intervention only on known trouble cases such as Obsoletes.
 
* Does the review differ depending on who is building the package (cla+1 vs in the packager group)?


* Do we trust the person whose repo/package was accepted into the Playground repository to keep it up-to-date and address serious bugs/security issues?
* Do we trust the person whose repo/package was accepted into the Playground repository to keep it up-to-date and address serious bugs/security issues?
Line 73: Line 104:
*** If a person misuses the community's trust by intentionally packaging malware or not fixing serious security issues found in his packages, then we could blacklist his FAS account which would prevent inclusion of his packages in the Playground repository.
*** If a person misuses the community's trust by intentionally packaging malware or not fixing serious security issues found in his packages, then we could blacklist his FAS account which would prevent inclusion of his packages in the Playground repository.


* Do we allow conflicts with packages in the main repo?
* Do we expect people to package stable/usable software in the Playground repository?
** If we would want to enforce the content of the repository to be something stable, then we would be back to approving things individually.
** Probably some packages will be more unstable than others and that's fine.
** We could at least put up some guidance that would promote the idea that the Playground repository should contain stable/usable software (similar to the [[Rawhide#Goals|first goal of the Rawhide repository]]) and that bleeding edge/"eats your babies" software should be rather put into a COPR (with a warning description next to it).


* Do we allow replacement of packages in the main repo?
* Builds from coprs or whole coprs might get deleted generally. Do we allow that in Playground repository? On [http://meetbot.fedoraproject.org/fedora-meeting/2014-04-15/env-and-stacks.2014-04-15-12.01.log.html meeting on 04/15] we discussed some POVs:
** Do we allow "backdoor replacement" of packages in the main repo?  ie: Let's say I have a package in the playground repo: NetworkManager2.1.  And that conflicts with NetworkManager. Is that allowed?  Is it allowed as long as it doesn't have any virtual provides/obsoletes that would automatically allow it to replace the package in the main repo?
** Allowing to delete those will save some space that is limited now
** With deleting we will lose possibility to get sources for the packages users can actually be using (a question if that is problem for GPL packages was raised -- 2014-04-15 Richard F. states it should be OK as soon as we delete SRPM at the same time as RPMs)


* Do we allow conflicts between packages in the Playground Repo?
=== Provide content of Coprs ===
 
* Use software written for softwarecollections.org or add fields into pkgdb2 (test instance http://209.132.184.188/) ?
* Do we allow replacement of other packages in the Playground Repository(How do we stop this in our implementation?)
** Display the package(s) in the repository
** Do we allow "backdoor replacement" in the playground repo?
** Display the date when the package was added
** Display the date when the package was last updated
** Display who owns the package, if a provenpackager a special symbol is displayed?
** Display the status of the package (like if it is bad, unusable, removed, terrific)
** Is package in Fedora, or moving to Fedora?  Which version.
** Is the package signedIf not signed do we want to state why?  Do we want to include a link to request that this package be considered for signing?
** Are there any tests associated with this package? - automated, manual, etc.
** Does package have broken dependencies? If so, it is marked a certain way.


* How do we deal with multiple versions of same package provided by different COPRs?
== Taskotron tests ==
Which tests do we want to run on Playground repo?


* Do we expect people to package stable/usable software in the Playground repository?
* Is it installable on latest Fedora with Playground repo? (no missing Requires).
** If we would want to enforce the content of the repository to be something stable, then we would be back to approving things individually.
* ...
** Probably some packages will be more unstable than others and that's fine.
** We could at least put up some guidance that would promote the idea that the Playground repository should contain stable/usable software (similar to the [[Rawhide#Goals|first goal of the Rawhide repository]]) and that bleeding edge/"eats your babies" software should be rather put into a COPR (with a warning description next to it).


== Problems ==
== Problems ==


=== 1 Big repo vs multiple small ones ===
=== SOLVED - 1 Big repo vs multiple small ones ===
decided - [http://meetbot.fedoraproject.org/fedora-meeting/2014-04-01/env-and-stacks.2014-04-01-12.02.log.html April 1, 2014]


Ideally users would enable just one "playgrond" repo and would get all nice updates. However this has several issues:
Ideally users would enable just one "playgrond" repo and would get all nice updates. However this has several issues:
Line 102: Line 144:
* Each project has a COPR repo (already done since that's how they are built)
* Each project has a COPR repo (already done since that's how they are built)
* Playground repo contains these repo files
* Playground repo contains these repo files
* We can add GUI support for enabling on per-feature basis (i.e. install playground repo for "Dajngo 1.6" or "Chromium" or ...)
* We can add GUI support for enabling on per-feature basis (i.e. install playground repo for "Django 1.6" or "Chromium" or ...)
* Possible conflicts are between features. It's not ideal but that way there *can* be conflicts and they are not catastrophic. People who want to test django do not necessarily want to test Chromium (or other way around)
* Possible conflicts are between features. It's not ideal but that way there *can* be conflicts and they are not catastrophic. People who want to test django do not necessarily want to test Chromium (or other way around)



Latest revision as of 19:10, 28 September 2016

Old page
This page has been marked as "old", and likely contains content that is irrelevant or incorrect. If you can, please update this page. This page will be deleted if action is not taken.


This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page. This document will eventually contain a roadmap for implementing the Playground repository. It will outline what the playground repository is and what groups, manpower, and other resources will be needed in order to implement it.

The Playground repository gives contributors a place to host packages that are not up to the standards of the main Fedora repository but may still be useful to other users. For now the Playground repository contains both packages that are destined for eventual inclusion into the main Fedora repositories and packages that are never going to make it there. Users of the repository should be willing to endure a certain amount of instability when using packages from here.

All packages in Playground must play nice - no bad licenses, no proprietary software, no patented software.

Description

Policies

  • Packages must follow the Legal Guidelines. In particular, the license for all packages must be approved in the Legal Guidelines.
  • Packages may violate other Fedora Packaging Guidelines.

How the repository works

Packages for the repository are built in COPR. The COPR owner can propose the repository as a whole for inclusion into the Playground repository by marking it as such in COPR. Repositories/packages successfully built and satisfying the Playground repository's Policies are copied into the Playgroud repository. The one Playground repository includes many Copr repositories.

  • How do the updates work?
    • The Playground repository follows the rolling release model. Data in Playground are more repository of repositories. All the builds in the COPR repositories that are selected to feed the Playground repository are marked as Playground repository content.
    • This is similar to the Rawhide repository.
    • Initially, the Bodhi update system will not be used.
    • These decisions were made on the March 4, 2014 meeting, but after working with dnf plugin was changed.
  • Does it have an additional testing repository?
    • Initially, there won't be an additional testing repository. If packagers want to provide some testing packages, they can create an additional COPR that will contain these testing packages.
    • This decision was made on the March 4, 2014 meeting.
  • Self hosting: All packages in the Playground repository are buildable using only packages in the Fedora' main repository and the Playground repository.
  • Upgrades and broken dependencies
    • FedUp will support upgrades with the packages in the Playground repository as if it was a generic third party repository. No special handling of the Playground repository will be implemented.
    • Packages in the Playground repository may have broken dependencies. We'll encourage maintainers to keep the package set free of these problems but by its nature (rolling, experimental, having packages who's dependencies change frequently) the Playground repository may have these problems more frequently than the Fedora's main repository.
    • We'll also implement a way to automatically notify the package owners about broken dependencies.
    • These decisions were made on the March 11, 2014 meeting
  • At the moment, the Playground repository is not multilib. This may change in the future if someone devotes the time to make it happen.
  • Reviews
    • We will provide a kind of "non-blocking review service" and later we will talk about whether we should make it mandatory or not. Coprs won't be blocked by reviews results in the beginning. April 1, 2014

Identified needs

Need How necessary Groups to Coordinate with
Disk space for the yum/dnf repositories Necessary Infra
Ability to mark an individual COPR for inclusion in the Playground repository Necessary Copr devs
Continuously regenerating repodata Necessary Infra
Daily composes of the Playground repository Necessary Infra
Copr deployment that's considered reliable enough to build packages for this repo Very nice to have Infra/Copr devs
Build from a git repository URL and revision hash Optional but nice to have Copr devs
deltarpms Optional but nice to have Releng
multilib Optional but nice to have Releng
Mirroring/mirrormanager Optional Infrastructure/releng
Signing by obs-signd Necessary Copr devels
Reviews - automatic service for reviews Optional pingou&sochotni

Open Questions

We'll need to answer these questions and by their answers, flesh out the Description and add additional work items to the Identified needs section.

  • Taskotron - basic sanity for inclusion into Playground?

Open Questions might be solved by multiple repos proposal

  • Do we allow conflicts with packages in the main repo?
  • Do we allow replacement of packages in the main repo?
    • Do we allow "backdoor replacement" of packages in the main repo? ie: Let's say I have a package in the playground repo: NetworkManager2.1. And that conflicts with NetworkManager. Is that allowed? Is it allowed as long as it doesn't have any virtual provides/obsoletes that would automatically allow it to replace the package in the main repo?
  • Do we allow conflicts between packages in the Playground Repo?
  • Do we allow replacement of other packages in the Playground Repository? (How do we stop this in our implementation?)
    • Do we allow "backdoor replacement" in the playground repo?
  • How do we deal with multiple versions of same package provided by different COPRs?
  • Do we trust the person whose repo/package was accepted into the Playground repository to keep it up-to-date and address serious bugs/security issues?
    • Just telling users that they should keep up with the security issues themselves is not a solution since that's well understood to be near impossible.
    • The problem with reactive removal of such packages from the repository is that this doesn't remove packages from users' systems.
      • Although the problem of package removal also exists in Fedora's main repos, it is mitigated somewhat since there we have a larger maintainer pool for addressing orphans, short lifecycle means that abandoned packages disappear in about a year, and packagers packaging things for the main repo have a higher bar to entry and are generally more serious and knowledgeable.
      • One possible solution is to set up an empty package that obsoletes such a problematic package.
      • Another solution is to have the fedora-playground-release package which has obsoletes like: Obsoletes: badapp-$version.
      • If a person misuses the community's trust by intentionally packaging malware or not fixing serious security issues found in his packages, then we could blacklist his FAS account which would prevent inclusion of his packages in the Playground repository.
  • Do we expect people to package stable/usable software in the Playground repository?
    • If we would want to enforce the content of the repository to be something stable, then we would be back to approving things individually.
    • Probably some packages will be more unstable than others and that's fine.
    • We could at least put up some guidance that would promote the idea that the Playground repository should contain stable/usable software (similar to the first goal of the Rawhide repository) and that bleeding edge/"eats your babies" software should be rather put into a COPR (with a warning description next to it).
  • Builds from coprs or whole coprs might get deleted generally. Do we allow that in Playground repository? On meeting on 04/15 we discussed some POVs:
    • Allowing to delete those will save some space that is limited now
    • With deleting we will lose possibility to get sources for the packages users can actually be using (a question if that is problem for GPL packages was raised -- 2014-04-15 Richard F. states it should be OK as soon as we delete SRPM at the same time as RPMs)

Provide content of Coprs

  • Use software written for softwarecollections.org or add fields into pkgdb2 (test instance http://209.132.184.188/) ?
    • Display the package(s) in the repository
    • Display the date when the package was added
    • Display the date when the package was last updated
    • Display who owns the package, if a provenpackager a special symbol is displayed?
    • Display the status of the package (like if it is bad, unusable, removed, terrific)
    • Is package in Fedora, or moving to Fedora? Which version.
    • Is the package signed? If not signed do we want to state why? Do we want to include a link to request that this package be considered for signing?
    • Are there any tests associated with this package? - automated, manual, etc.
    • Does package have broken dependencies? If so, it is marked a certain way.

Taskotron tests

Which tests do we want to run on Playground repo?

  • Is it installable on latest Fedora with Playground repo? (no missing Requires).
  • ...

Problems

SOLVED - 1 Big repo vs multiple small ones

decided - April 1, 2014

Ideally users would enable just one "playgrond" repo and would get all nice updates. However this has several issues:

  • We'd need support in rel-eng for multiple versions of identical package (problems with composes)
  • Users would get *all* playground packages not just ones they are interested in
  • There is no way to specify which packages from playground to install (or they are inadequate)

Most likely better approach is repo-of-repos where:

  • Each project has a COPR repo (already done since that's how they are built)
  • Playground repo contains these repo files
  • We can add GUI support for enabling on per-feature basis (i.e. install playground repo for "Django 1.6" or "Chromium" or ...)
  • Possible conflicts are between features. It's not ideal but that way there *can* be conflicts and they are not catastrophic. People who want to test django do not necessarily want to test Chromium (or other way around)