From Fedora Project Wiki

(Created this page to analyze the requirements of our TCMS)
 
(Add some items and use table format)
 
(2 intermediate revisions by the same user not shown)
Line 4: Line 4:
This proposal analyses our current mediawiki-based system workflow and use cases, list the pro's and con's and compare them with nitrate system. The purpose is to find out what features we Must-Have and Nice-to-have in future TCMS, aka nitrate, and nitrate should be customized bases on these requirements.  
This proposal analyses our current mediawiki-based system workflow and use cases, list the pro's and con's and compare them with nitrate system. The purpose is to find out what features we Must-Have and Nice-to-have in future TCMS, aka nitrate, and nitrate should be customized bases on these requirements.  


== Current wiki-based TCMS ==
== wiki-based TCMS VS Nitrate System ==




== pro's and con's of wiki-based workflow ==
{| class="wikitable sortable"
=== Pro's ===
|-  
! width="50%"| Wiki !! width="50%"| Nitrate
|-
! colspan="2"| Use cases 
|-
|
General Manual Test Use case：


* Easy to edit
For each release:
* Permit anonymous testers
* History roll back, easy recovery
* multi-testers contribute one case


Details:
* Create test plan
* Only one result page for current installation/desktop validation event.
* (Update Release Criteria)
* Search from categories or Create new test cases
* Create test result template


=== Con's ===
Then for each build:


* Results tracking
* Create test result page
* Results querying
* Use redirect link as current links
* Data statistics
* Send announcement
* Appearance
* Execute tests and provide test results
* Syntax editing without form
* Summarize the report manually
|
Manual testing Use Case:


== Must-Have and Nice-to-Have ==
# QE Project/Team Lead assigns feature to be tested
# QA searches TCMS for Test Plan
# QA creates new Test Run
# QA Executes Test Run
# Test Run report available for viewing


Writing a Test Plan Use Case:


 
# QE Project/Team Lead assigns feature to be tested
== Nitrate TCMS Features==
# QA writes a new Test Plan
 
# QA adds Test Cases:
* Tree view - automatically arrange plan, sub-plan, nodes, sub-nodes.
## Create new Test Case
* Email Notification for work flow.
## Import XML Test Case
* Status easily adjusted for tracking 
## Add existing Test Case
## QA Executes Test Run
|-
! colspan="2"| {{result|pass|Pros}}
|-
| <!-- Pros for wiki-->
* Easy to edit/modify
* Support anonymous feedback
* Integrated with FAS
* Supports a decent API for extracting content and useful for metrics gathering
* History roll back, easy recovery
* Multi-testers contribute one case
* Low barriers to entry
* Flexibility
* User can apply for different permission
* Pages with different name space have diff permission. 
* Different name spaces and categories to organize pages.
* Templates designed to fit for different instance.
<!-- Details:
* Only one result page for current installation/desktop validation event.-->
| <!-- Pros for Nitrate-->
* Tree view showing the current plan, and its parents and children using a tree style layout. It provides the ability to edit both parent and child plans. .
* Email Notification in workflow.
* Runs and cases easily cloned along with more additional options
* Runs and cases easily cloned along with more additional options
* Bug list automatically generated
* Bug list automatically generated
* Report automatically generated
* Test run report auto-generated
* Test cases have reviewing status
Limitations:
* Different status of a Test Run for tracking
* More than one tag for a test case
|-
! colspan="2"| {{result|fail|Cons}}
|-
| <!-- Cons for wiki -->
* Results tracking
* Results querying
* Data statistics
* Simple appearance
* Syntax editing without forms
* Cases without a category existed
* Cases have no 'review' phase
| <!-- Cons for Nitrate -->
* Users without authentication (Guests) are granted read only permission to Test Cases and Test Plans
* multi-testers contribute to one case not supported?
* no history roll back
* permissions for users are not well grouped
|-
|}


* Test runs/cases need be assigned to specific persons.
== Must-Have and Nice-to-Have ==
* single result permitted to submit for each run.
<<TBD>>
   
   
== Migration work ==
== Migration work ==
<<TBD>> <!--how the crowd-source test workflow would integrate within tcms.-->

Latest revision as of 10:06, 16 December 2010

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Overview

This proposal analyses our current mediawiki-based system workflow and use cases, list the pro's and con's and compare them with nitrate system. The purpose is to find out what features we Must-Have and Nice-to-have in future TCMS, aka nitrate, and nitrate should be customized bases on these requirements.

wiki-based TCMS VS Nitrate System

Wiki Nitrate
Use cases

General Manual Test Use case：

For each release:

  • Create test plan
  • (Update Release Criteria)
  • Search from categories or Create new test cases
  • Create test result template

Then for each build:

  • Create test result page
  • Use redirect link as current links
  • Send announcement
  • Execute tests and provide test results
  • Summarize the report manually

Manual testing Use Case:

  1. QE Project/Team Lead assigns feature to be tested
  2. QA searches TCMS for Test Plan
  3. QA creates new Test Run
  4. QA Executes Test Run
  5. Test Run report available for viewing

Writing a Test Plan Use Case:

  1. QE Project/Team Lead assigns feature to be tested
  2. QA writes a new Test Plan
  3. QA adds Test Cases:
    1. Create new Test Case
    2. Import XML Test Case
    3. Add existing Test Case
    4. QA Executes Test Run
Pass pass Pros
  • Easy to edit/modify
  • Support anonymous feedback
  • Integrated with FAS
  • Supports a decent API for extracting content and useful for metrics gathering
  • History roll back, easy recovery
  • Multi-testers contribute one case
  • Low barriers to entry
  • Flexibility
  • User can apply for different permission
  • Pages with different name space have diff permission.
  • Different name spaces and categories to organize pages.
  • Templates designed to fit for different instance.
  • Tree view showing the current plan, and its parents and children using a tree style layout. It provides the ability to edit both parent and child plans. .
  • Email Notification in workflow.
  • Runs and cases easily cloned along with more additional options
  • Bug list automatically generated
  • Test run report auto-generated
  • Test cases have reviewing status
  • Different status of a Test Run for tracking
  • More than one tag for a test case
Fail fail Cons
  • Results tracking
  • Results querying
  • Data statistics
  • Simple appearance
  • Syntax editing without forms
  • Cases without a category existed
  • Cases have no 'review' phase
  • Users without authentication (Guests) are granted read only permission to Test Cases and Test Plans
  • multi-testers contribute to one case not supported?
  • no history roll back
  • permissions for users are not well grouped

Must-Have and Nice-to-Have

<<TBD>>

Migration work

<<TBD>>

Retrieved from "https://fedoraproject.org/w/index.php?title=Rhe/tcms_requirements_proposal&oldid=211801"