m (Tflink moved page User:Tflink/f21 delay taskbot development proposal to User:Tflink/f21 delay taskotron development proposal: project name change due to potential trademark/copyright issues) |
(changing taskbot references to taskotron and added admon/note explaining reasons) |
||
| Line 1: | Line 1: | ||
= Fedora 21 Delay Proposal: | = Fedora 21 Delay Proposal: taskotron Development = | ||
== Summary == | == Summary == | ||
While progress on automated testing has been slow as of late, the changes proposed for [[Fedora.next]] will be made much more workable with better automation in the QA space. Instead of working on small improvements as time allows between Fedora releases, we propose that Fedora 21 be delayed so that we can focus on getting our next generation automation system in place to support the new direction for Fedora. | While progress on automated testing has been slow as of late, the changes proposed for [[Fedora.next]] will be made much more workable with better automation in the QA space. Instead of working on small improvements as time allows between Fedora releases, we propose that Fedora 21 be delayed so that we can focus on getting our next generation automation system in place to support the new direction for Fedora. | ||
{{admon/note|Name Change|Due to potential trademark/copyright conflicts, we have re-named the project as Taskotron}} | |||
== Owner == | == Owner == | ||
| Line 39: | Line 41: | ||
== Background == | == Background == | ||
While [[AutoQA]] has served us well over the years, it has some issues that we need to fix in order to move forward. With this in mind, we have proposed a new automation system which we are calling | While [[AutoQA]] has served us well over the years, it has some issues that we need to fix in order to move forward. With this in mind, we have proposed a new automation system which we are calling taskotron. We will be preserving as much of AutoQA as makes sense but moving forward is a higher priority than reusing old code and solutions. | ||
* [http://tirfa.com/an-initial-idea-for- | * [http://tirfa.com/an-initial-idea-for-taskotron.html An initial description of taskotron] | ||
* [http://tirfa.com/how-is- | * [http://tirfa.com/how-is-taskotron-different-from-autoqa.html How is taskotron different from AutoQA] | ||
The [[User:Tflink/ | The [[User:Tflink/taskotron_development_plan|development plan for taskotron]] describes several phases for incremental development of automation features. [[User:Tflink/taskotron_development_subprojects#Potential_Future_System_Checks|The full list of taskotron sub-projects is also available]]. [[User:Tflink/taskotron_development_plan#Phase_0:_Investigation_and_Preparation|taskotron Phase 0]] is currently in progress and unless there are significant disruptions from Fedora 20 testing, will be complete before the end of January 2014. | ||
== Detailed Description == | == Detailed Description == | ||
| Line 73: | Line 75: | ||
=== No Delay === | === No Delay === | ||
* Completion of [[User:Tflink/ | * Completion of [[User:Tflink/taskotron_development_plan#Phase_0:_Investigation_and_Preparation|taskotron Phase 0]] | ||
* Completion of [[User:Tflink/ | * Completion of [[User:Tflink/taskotron_development_plan#Phase_1:_AutoQA_Replacement|taskotron Phase 1]] by May 1, 2014. | ||
* Starting later phases after completion of phase 1. | * Starting later phases after completion of phase 1. | ||
=== 1 Month Delay === | === 1 Month Delay === | ||
* Completion of [[User:Tflink/ | * Completion of [[User:Tflink/taskotron_development_plan#Phase_0:_Investigation_and_Preparation|taskotron Phase 0]] | ||
* Completion of [[User:Tflink/ | * Completion of [[User:Tflink/taskotron_development_plan#Phase_1:_AutoQA_Replacement|taskotron Phase 1]] by March 1, 2014. | ||
* Starting later phases after completion of phase 1. | * Starting later phases after completion of phase 1. | ||
=== 2-3 Month Delay === | === 2-3 Month Delay === | ||
After the completion of [[User:Tflink/ | After the completion of [[User:Tflink/taskotron_development_plan#Phase_1:_AutoQA_Replacement|taskotron Phase 1]], there are multiple options for more functionality to implement (Later phases in [[User:Tflink/taskotron_development_subprojects|the development plan]], other [[User:Tflink/taskotron_development_subprojects|subprojects]]) but with the exception of implementing an ABI break/change tool (less than 1 month), there are too many unknowns for us to make reasonable estimates on the time required to complete those phases. | ||
We do think it is reasonably likely that we'd be able to complete either [[User:Tflink/ | We do think it is reasonably likely that we'd be able to complete either [[User:Tflink/taskotron_development_plan#Phase_X:_Beaker_Integration|taskotron Phase X]], [[User:Tflink/taskotron_development_plan#Phase_Y:_Installation_Checks|taskotron Phase Y]] or a subset of [[User:Tflink/taskotron_development_plan#Phase_Z:_More_Sophisticated_Checks|taskotron Phase Z]] given an additional 2 months (making for a 3 month total delay) but we are not willing to commit to their completion in that time frame until more of [[User:Tflink/taskotron_development_plan#Phase_1:_AutoQA_Replacement|taskotron Phase 1]] has been implemented. | ||
== User Experience == | == User Experience == | ||
| Line 101: | Line 103: | ||
== Documentation == | == Documentation == | ||
Documentation is still being written but so far, we have: | Documentation is still being written but so far, we have: | ||
* [http://tirfa.com/an-initial-idea-for-taskbot.html An initial description of | * [http://tirfa.com/an-initial-idea-for-taskbot.html An initial description of taskbot] | ||
* [http://tirfa.com/how-is-taskbot-different-from-autoqa.html How is | * [http://tirfa.com/how-is-taskbot-different-from-autoqa.html How is taskbot different from AutoQA] | ||
* [[User:Tflink/ | * [[User:Tflink/taskotron_development_subprojects|taskotron Subprojects]] | ||
* [[User:Tflink/ | * [[User:Tflink/taskotron_development_plan|Development plan for taskotron]] | ||
Latest revision as of 20:32, 11 November 2013
Fedora 21 Delay Proposal: taskotron Development
Summary
While progress on automated testing has been slow as of late, the changes proposed for Fedora.next will be made much more workable with better automation in the QA space. Instead of working on small improvements as time allows between Fedora releases, we propose that Fedora 21 be delayed so that we can focus on getting our next generation automation system in place to support the new direction for Fedora.
Due to potential trademark/copyright conflicts, we have re-named the project as Taskotron
Owner
- Name: Tim Flink
- Email: tflink at fedoraproject org
- Release notes owner:
Current status
- Targeted release: Fedora 21
- Last updated: October 7, 2013
- Tracker bug: None
Assumptions
There are very few specifics on the timeframe for the proposals requested by fesco for projects which could be done during if f21 was delayed for infrastructure/tooling work in qa, releng and other areas. Therefore, I am making several assumptions to make this proposal clearer.
- Any delay would start after January 31, 2014
- Available people is still TBD - this will depend on any other proposals considered and accepted by FESCo for the possible delay period
- I assume that there will be at least 3 full time engineers dedicated to this work. There may be additional folks from the community but that part is still TBD as I haven't asked for commitments from anyone else yet.
- Conservative estimates that under promise and over deliver are preferred.
- There are still unknowns in this plan so I am making conservative estimates and leaving a wishlist of things to work on if time permits
Background
While AutoQA has served us well over the years, it has some issues that we need to fix in order to move forward. With this in mind, we have proposed a new automation system which we are calling taskotron. We will be preserving as much of AutoQA as makes sense but moving forward is a higher priority than reusing old code and solutions.
The development plan for taskotron describes several phases for incremental development of automation features. The full list of taskotron sub-projects is also available. taskotron Phase 0 is currently in progress and unless there are significant disruptions from Fedora 20 testing, will be complete before the end of January 2014.
Detailed Description
We have been planning and starting the investigation process for a new automation system which will eventually replace AutoQA. Unfortunately, this is far from a small undertaking and most of our resources are occupied with testing during the normal Fedora release cycle. There is some time between releases but it has proven to be far shorter than we would need to get a significant project started and finished before the next release.
By delaying the release of Fedora 21, we would be able to commit more resources to automation development and the work would be done more quickly. Exactly how quickly is addressed with scope.
Benefit to Fedora
The main benefit to Fedora would be more test coverage stemming from more automated checking and freeing up human testers to do more complicated tests than currently done as part of release validation. This new automation is coming whether the delay is accepted or not but the delay would free up more developers to work on implementation, bringing these benefits to Fedora faster.
Benefit To End Users
- More stability in Fedora
Benefit To Developers
- More automated checks
- Better communication of results
- Opportunity to have checks added to automation
Benefit To Testers
- Less time spent on the validation treadmill doing repetitive tasks
- More time to participate in test days, test new features
Scope
The scope of this proposal depends on the length of the delay being considered.
No Delay
- Completion of taskotron Phase 0
- Completion of taskotron Phase 1 by May 1, 2014.
- Starting later phases after completion of phase 1.
1 Month Delay
- Completion of taskotron Phase 0
- Completion of taskotron Phase 1 by March 1, 2014.
- Starting later phases after completion of phase 1.
2-3 Month Delay
After the completion of taskotron Phase 1, there are multiple options for more functionality to implement (Later phases in the development plan, other subprojects) but with the exception of implementing an ABI break/change tool (less than 1 month), there are too many unknowns for us to make reasonable estimates on the time required to complete those phases.
We do think it is reasonably likely that we'd be able to complete either taskotron Phase X, taskotron Phase Y or a subset of taskotron Phase Z given an additional 2 months (making for a 3 month total delay) but we are not willing to commit to their completion in that time frame until more of taskotron Phase 1 has been implemented.
User Experience
Fedora users may notice the delayed release and will hopefully notice an increase in quality.
Dependencies
There are no dependencies that are not listed in this proposal.
Contingency Plan
We keep working on automation with more limited resources. The scope of our plans will remain the same but our timelines will stretch out without the resources we would otherwise gain from delaying Fedora 21.
Documentation
Documentation is still being written but so far, we have: