Tcms Comparison

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

This page objectively compares feature sets of Wiki and Nitrate systems in relation to the important workflows. Instead of displaying a long feature lists, it is divided into groups by use cases. To avoid overlaps, features compared in former use cases are not listed in later cases again.

Key Section

Icon	Explanation
pass	Supported
fail	Unsupported currently
Unknown unknown	Unknown/Research needed
warn	Partly supported
P1	Must-Have missing Feature in Nitrate
P2	Nice-To-Have missing Feature in Nitrate

Wiki vs Nitrate Feature Comparison

Creating A Generic Page

Priority Feature fedoraproject.org/wiki nitrate Any page creation pass fail Can only create test case/plan/run P2 Readable link names pass human-readable link names fail use ID for identification Data entry format pass mediawiki markup pass tinyMCE Moving a Page pass by redirect links pass by cloning or changing the summary(name) of case/plan/run Templates creation/use pass such as Special:MostLinkedTemplates fail Editing preview pass warn 'save and continue editing' Supports data entry using a form fail wiki syntax pass P2 Discussion page pass Talk page fail Can only add comments in test cases P1 History rollback(Undo changes) pass fail P1 History comparison pass fail History logs pass pass Only offer change logs and text version history of plan and case Subpage using pass Unknown unknown by Tree View? Bookmarks/watchlist pass watch the page pass bookmark the page

Creating A Test Case

Priority Feature fedoraproject.org/wiki nitrate Renaming a case warn move to another page pass change the summary Test case format template pass Template:QA/Test_Case pass Defined Linking files/packages/images pass Upload somewhere and link to it pass Test cases/plans support attachments or link to wiki Case draft status warn manually add draft category/note pass Set as PROPOSED or NEED_UPDATE Case review status fail approved in ticket or somewhere else pass Set as CONFIRMED Categorizing cases pass manually add to category pass add category or use tag or add to plan to group cases P2 Categories creation pass user can create it with certain permission fail cases categories can only be added by Admin P1 Description part in test case pass warn write in 'Notes' part, where syntax editing is needed. Tests that impact multiple packages warn Categories could be used to organize this data, but get messy pass allows linking tests to the packages they are designed to test Test case re-use (write once, link anywhere) pass search from Category and link to it pass test runs re-use cases from a plan, plans clone them from other plans P2 The time historical test case results be available pass all history? Unknown unknown unknown P2 Re-use old versioned test case pass by linking to its history fail can view text history and version only

Creating A Test Plan

Priority Feature fedoraproject.org/wiki nitrate Test Plan re-use/update pass copy contents from previous pass clone previous plans Linking test cases pass link to cases section on test result template page pass clone from other plans/create new tests P2 Plan draft status warn manually add draft category/note fail Set as non-active? P2 Plan review status warn manually add reviewer and review time fail Set as active? Categorizing plans pass manually add to category pass no category for plan but have Plan Type and Tag etc.

Creating A Test Result Page(Test Run) Template
Priority Feature fedoraproject.org/wiki nitrate Result Page(Test Run) Template pass fail doesn't support template creation. Test run(result page) can be cloned from previous run with modifications. Sortable table pass pass using sort Collapse table warn hard to manage as a whole pass P1 Group cases (by media) pass flexible wiki design fail diff test runs;sorting;diff tags

Creating A Test Result Page(Test Run)

Priority Feature fedoraproject.org/wiki nitrate Test result page(Test run) creation pass warn Create from plans P1 documents in test result page pass flexible wiki markup design warn write in 'Notes' field without any syntax P1 Moving test results pass manually copy fail manually add one by one? P2 Directing to current test run pass redirect links to current result page fail set test run as running or finished Categorizing runs/result pages pass manually add to category pass Can set 'product', 'version' and 'tag' etc

Posting Test Results

Priority Feature fedoraproject.org/wiki nitrate P2 Quick access to active test runs pass Open 'current' links fail search the run's ID or search my runs(assigned to me in advance) P1 Result format pass use Template:Result warn Idle, Passed, Failed, Running, Paused, Blocked, Error Comment as references pass pass Bug posting pass use Template:Bz pass P1 Multiple contributions for each case pass fail only display the last modified one and logs *Signatures using pass ~~~~ -> Rhe 09:59, 18 January 2011 (UTC) fail

Searching Test Results/Cases/Plans
Priority Feature fedoraproject.org/wiki nitrate Searching cases/plans/runs fail only by searching from categories or name pass advanced search Toolbox(what links to here) pass pass plans, cases, and runs related to each other

Uploading Files
Priority Feature fedoraproject.org/wiki nitrate P2 Upload files/packages/images pass upload somewhere and link to it warn test cases/plans support attachments, test runs don't P2 Upload updates.img/repo to certain server pass fedorapeople server fail link to wiki.

Generating A Test Summary
Priority Feature fedoraproject.org/wiki nitrate Result summary/report generation fail run curl command pass automaticly

Administrating

Priority Feature fedoraproject.org/wiki nitrate P1 Authorities for pages pass Use namespaces Unknown unknown Can admin configure it? Supporting anonymous user read-only access pass pass P1 Supporting anonymous user read-write access pass fail Feature extension pass pass P1 Page protection pass by Admin Unknown unknown Case/plan/run deletion pass by Admin pass Set as DISABLED. Test run can be deleted. Licensing pass pass P1 License the content pass Unknown unknown should match wiki Integration with FAS pass Unknown unknown P2 Integration between update tools (bodhi/f-e-k) and our test infrastructure. pass investigating Unknown unknown P1 Upstream project community pass upstream wiki list warn upstream nitrate list

Test Days

Priority Feature fedoraproject.org/wiki nitrate Test day main/intro page pass warn link to wiki Test day SOP guide page pass warn link to wiki Test day/event result template page pass fail Clone from previous test run Test day schedule pass warn link to wiki P1 Test day page(run) creation pass fail test runs can only be created from plans P1 Test day page documented contents pass fail write all intro to 'Notes' field will be too crowded! P2 Test day notification fail send out announcement manually warn can assign through mail, but mail content is little.

Release Validation Test Event

Priority Feature fedoraproject.org/wiki nitrate Test event Main/intro page pass warn link to wiki Test event SOP guide page pass warn link to wiki Test event Result template page pass fail Clone from previous test run. Release Criteria page pass warn link to wiki P1 Test cases priority fail manually add priority before each case link warn from P1 to P5, need change to: Alpha, Beta, Final. P1 Each case with different platforms pass fail two test runs for both platforms or clone each case for another platform Test result page(Test run) creation pass pass created from plans P1 Test result page documented contents pass warn write intro to 'Notes' field without any syntax P2 Test event notification fail send out announcement manually warn can assign through mail, but the content is little. Progress bar for each run fail pass