Tcms Comparison: Difference between revisions

Revision as of 08:55, 20 January 2011

This page objectively compares feature sets of Wiki and Nitrate systems in relation to the important workflows. Instead of displaying a long feature lists, it is divided into groups by use cases. Click [show] button on each item to view the detailed comparison. To avoid overlaps, features compared in former use cases are not listed in later cases again.

Creating A Common Page
Feature	fedoraproject.org/wiki	nitrate
Any page creation	pass	fail Can only create test case/plan/run
Readable link names	pass human-readable link names	fail use ID for identification
Data entry format	mediawiki markup	tinyMCE
Moving a Page	pass by redirect links	pass by cloning or changing the summary(name) of case/plan/run
Templates creation/use	pass such as Special:MostLinkedTemplates	fail
Editting preview	pass	fail
Supports data entry using a form	fail wiki syntax	pass
Talk page	pass	warn Can only add comments in test cases
History rollback	pass	fail Only offer change logs
Changes comparison	pass	warn Only offer change logs
Categories creation	pass manually add new category	warn cases categories can be added by Admin
Adding pages to categories	pass manually add category	warn Only cases have categories
Subpage using	pass	Unknown unknown by Tree View?
Bookmarks/watchlist	pass watch the page	pass bookmark the page

Creating A Test Case
Feature	fedoraproject.org/wiki	nitrate
Renaming a case	warn move to another page	pass change the summary
Test case template	pass Template:QA/Test_Case	pass Defined
Linking files/packages/images	pass Upload somewhere and link to it	pass Test cases/plans support attachments
Case draft status	fail manually add draft category/note	pass Set as PROPOSED or NEED_UPDATE
Case review status	fail approved in ticket or somewhere else	pass Set as CONFIRMED
Categorizing cases	pass manually add to category	pass add category or use tag or add to plan to group cases
Description part in test case	pass	fail only have setup and notes parts
Tests that impact multiple packages	warn Categories could be used to organize this data, but get messy	pass allows linking tests to the packages they are designed to test
Test case re-use (write once, link anywhere)	pass search from Category and link to it	pass clone from other plans

Creating A Test Plan
Feature	fedoraproject.org/wiki	nitrate
Test Plan re-use/update	pass copy contents from previous	pass clone previous plans
Linking test cases	pass linking to cases section on test result template page	pass clone from other plans
Plan draft status	fail manually add draft category/note	fail Set as non-active?
Plan review status	fail manually add reviewer and review time	fail Set as active
Categorizing plans	pass manually add to category	pass no category for plan but have Plan Type and Tag etc.

Creating A Test Result Page Template
Feature	fedoraproject.org/wiki	nitrate
Result Page Template	pass	fail doesn't support result page template creation. Test run(result page) can be cloned from previous run with modifications.
Sortable table	pass	pass using sort
Collapse table	warn hard to manage as a whole	pass
Group cases (by media)	pass flexible wiki design	fail

Creating A Test Result Page
Feature	fedoraproject.org/wiki	nitrate
Test result page creation	pass	warn Create from plans
Contents in test result page	pass flexible wiki markup design	warn write in 'Notes' field without any syntax except cases
Moving test results	pass manually copy	fail manually add one by one?
Directing to current test run	pass redirect links to current result page	fail set test run as running or finished
Categorizing runs/result pages	pass manually add to category	pass Can set 'product', 'version' and 'tag' etc

Posting Test Results
Feature	fedoraproject.org/wiki	nitrate
Quick access to active test runs	pass Open 'current' links	fail search the run's ID or search my runs(assigned to me in advance)
Result format	pass use Template:Result	pass defined
Comment as references	pass	pass
Bug posting	pass use Template:Bz	pass
Multiple contributions for each case	pass	fail only display the last modified one
*Signatures using	pass ~~~~ -> Rhe 09:59, 18 January 2011 (UTC)	fail

Searching Test Results/Cases/Plans
Feature	fedoraproject.org/wiki	nitrate
Searching cases/plans/runs	fail only by searching from categories or name	pass advanced search
Toolbox(what links to here)	pass	pass plans, cases, and runs related to each other

Uploading Files
Feature	fedoraproject.org/wiki	nitrate
Upload files/packages/images	pass upload somewhere and link to it	warn test cases/plans support attachments(test run doesn't)
Upload updates.img/repo to certain server	pass fedorapeople server	fail link to wiki.

Generating A Test Summary
Feature	fedoraproject.org/wiki	nitrate
Result summary/report generation	fail run curl command	pass automatic

Administrating
Feature	fedoraproject.org/wiki	nitrate
Authorities for pages	pass Use namespaces	Unknown unknown Can admin configure it?
Supporting anonymous user read-only access	pass	pass
Supporting anonymous user read-write access	pass	fail
Integration with FAS	pass	Unknown unknown
Feature extension	pass Add plugin	Unknown unknown Add plugin?
Page protection	pass by Admin	Unknown unknown
Case/plan deletion	pass by Admin	pass Set as DISABLED. Test run can be deleted.
License	green	green

Test Days
Feature	fedoraproject.org/wiki	nitrate
Test day main/intro page	pass	fail link to wiki.
Test day SOP guide page	pass	fail link to wiki.
Test day/event result template page	pass	fail Clone from previous test run.
Test day schedule	pass	fail link to wiki.
Test day page creation	pass	fail test runs can only be created from plans
Test day page contents	pass	fail write all intro to 'Notes' field will be too crowded!
Test day notification	fail send out announcement manually	warn Assign through mail, but the content is very brief.

Release Validation Test Event
Feature	fedoraproject.org/wiki	nitrate
Test event Main/intro page	pass	fail link to wiki.
Test event SOP guide page	pass	fail link to wiki.
test event Result template page	pass	fail Clone from previous test run.
Release Criteria page	pass	fail link to wiki.
Test cases priority	fail manually add priority before each case link	warn from P1 to P5, need change to: Alpha, Beta, Final.
Each case with different platforms	pass	fail ?? two test runs for both platforms?
Test result page creation	pass	pass created from plans
Test result page contents	pass	warn write intro to 'Notes' field without any syntax
Progress bar for each run	fail	pass

@@ Line 11: / Line 11: @@
 |style="background-color:tomato;"| {{result|fail}} Can only create test case/plan/run
 |-
-| Readable link names
+| style="color:purple"|Readable link names
 |style="background-color:lightgreen;"| {{result|pass}} human-readable link names
 |style="background-color:tomato;"| {{result|fail}} use ID for identification
@@ Line 23: / Line 23: @@
 |style="background-color:lightgreen;"| {{result|pass}} by cloning or changing the summary(name) of case/plan/run
 |-
-| Templates creation/use
+| style="color:purple"|Templates creation/use
 |style="background-color:lightgreen;"| {{result|pass}} such as [[Special:MostLinkedTemplates]]
 |style="background-color:tomato;"| {{result|fail}}
 |-
-| Editting preview
+| style="color:purple"|Editting preview
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|fail}}
 |-
-| Supports data entry using a form
+| style="color:purple"|Supports data entry using a form
 |style="background-color:tomato;"| {{result|fail}} wiki syntax
 |style="background-color:lightgreen;"| {{result|pass}}
 |-
-| Talk page
+| style="color:purple"|Talk page
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|warn}} Can only add comments in test cases
 |-
-| History rollback
+| style="color:red"|History rollback
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|fail}} Only offer change logs
 |-
-| Changes comparison
+| style="color:red"|Changes comparison
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|warn}} Only offer change logs
@@ Line 71: / Line 71: @@
 |-
 | Renaming a case
-|style="background-color:tomato;"| {{result|fail}} move to another page
+|style="background-color:tomato;"| {{result|warn}} move to another page
 |style="background-color:lightgreen;"| {{result|pass}}change the summary
 |-
@@ Line 94: / Line 94: @@
 |style="background-color:lightgreen;"| {{result|pass}} add category or use tag or add to plan to group cases
 |-
-| Description part in test case
+| style="color:red"|Description part in test case
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|fail}} only have setup and notes parts
@@ Line 121: / Line 121: @@
 |style="background-color:lightgreen;"| {{result|pass}} clone from other plans
 |-
-| Plan draft status
+| style="color:purple"|Plan draft status
 |style="background-color:tomato;"| {{result|fail}} manually add draft category/note
 |style="background-color:tomato;"| {{result|fail}} Set as non-active?
 |-
-| Plan review status
+| style="color:purple"|Plan review status
 |style="background-color:tomato;"| {{result|fail}} manually add reviewer and review time
 |style="background-color:tomato;"| {{result|fail}} Set as active
@@ Line 152: / Line 152: @@
 |style="background-color:lightgreen;"| {{result|pass}}
 |-
-| Group cases (by media)
+| style="color:purple"|Group cases (by media)
 |style="background-color:lightgreen;"| {{result|pass}}flexible wiki design
 |style="background-color:tomato;"| {{result|fail}}
@@ Line 163: / Line 163: @@
 ! width="20%"|Feature !! width="40%"|fedoraproject.org/wiki !! width="40%"|nitrate
 |-
-| Test result page creation
+| style="color:purple"|Test result page creation
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:lightgreen;"| {{result|warn}} Create from plans
 |-
-| Contents in test result page
+| style="color:red"|Contents in test result page
 |style="background-color:lightgreen;"| {{result|pass}} flexible wiki markup design
 |style="background-color:tomato;"| {{result|warn}} write in 'Notes' field without any syntax except cases
 |-
-| Moving test results
+| style="color:red"|Moving test results
 |style="background-color:lightgreen;"| {{result|pass}} manually copy
 |style="background-color:tomato;"| {{result|fail}} manually add one by one?
 |-
-| Redirecting links
+| style="color:purple"|Directing to current test run
 |style="background-color:lightgreen;"| {{result|pass}} redirect links to current result page
 |style="background-color:tomato;"| {{result|fail}} set test run as running or finished
@@ Line 190: / Line 190: @@
 ! width="20%"|Feature !! width="40%"|fedoraproject.org/wiki !! width="40%"|nitrate
 |-
-| Quick access to active test runs
+| style="color:purple"|Quick access to active test runs
 |style="background-color:lightgreen;"| {{result|pass}} Open 'current' links
 |style="background-color:tomato;"| {{result|fail}} search the run's ID or search my runs(assigned to me in advance)
@@ Line 206: / Line 206: @@
 |style="background-color:lightgreen;"| {{result|pass}}
 |-
-| Multiple contributions for each case
+| style="color:red"|Multiple contributions for each case
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|fail}} only display the last modified one
@@ Line 240: / Line 240: @@
 |style="background-color:lightgreen;"| {{result|warn}} test cases/plans support attachments(test run doesn't)
 |-
-| Upload updates.img/repo to certain server
+| style="color:purple"|Upload updates.img/repo to certain server
 |style="background-color:lightgreen;"| {{result|pass}} fedorapeople server
 |style="background-color:tomato;"| {{result|fail}} link to wiki.
@@ Line 262: / Line 262: @@
 ! width="20%"|Feature !! width="40%"|fedoraproject.org/wiki !! width="40%"|nitrate
 |-
-| Authorities for pages
+| style="color:red"|Authorities for pages
 |style="background-color:lightgreen;"| {{result|pass}} Use namespaces
 |style="background-color:white;"| {{result|unknown}} Can admin configure it?
@@ Line 270: / Line 270: @@
 |style="background-color:lightgreen;"| {{result|pass}}
 |-
-| Supporting anonymous user read-write access
+| style="color:purple"|Supporting anonymous user read-write access
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|fail}}
@@ Line 282: / Line 282: @@
 |style="background-color:white;"| {{result|unknown}} Add plugin?
 |-
-| Page protection
+| style="color:red"|Page protection
 |style="background-color:lightgreen;"| {{result|pass}} by Admin
 |style="background-color:white;"| {{result|unknown}}
@@ Line 317: / Line 317: @@
 |style="background-color:tomato;"| {{result|fail}} link to wiki.
 |-
-| Test day page creation
+| style="color:red"|Test day page creation
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|fail}} test runs can only be created from plans
 |-
-| Test day page contents
+| style="color:red"|Test day page contents
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|fail}} write all intro to 'Notes' field will be too crowded!
@@ Line 352: / Line 352: @@
 |style="background-color:tomato;"| {{result|fail}} link to wiki.
 |-
-| Test cases priority
+| style="color:red"|Test cases priority
 |style="background-color:tomato;"| {{result|fail}} manually add priority before each case link
 |style="background-color:lightgreen;"| {{result|warn}} from P1 to P5, need change to: Alpha, Beta, Final.
 |-
-| Each case with different platforms
+| style="color:red"|Each case with different platforms
 |style="background-color:lightgreen;"| {{result|pass}}
 |style="background-color:tomato;"| {{result|fail}} ?? two test runs for both platforms?

Search

Tcms Comparison: Difference between revisions

Revision as of 08:55, 20 January 2011