FedoraCommunity/UsabilityTestingRound1/AnalysisAndTakeaways

Remarks By Topic

Visibility

• (-1) Tester #1, a new packager, had never heard of or used Fedora Community before the call for usability testers.
• (-1) Tester #2, an experienced packager, had never heard of or used Fedora Community before the call for usability testers.

Branding

• (-1) Tester #1 thought the picture of Kyle was a little off-putting :)
• (-1) Tester #2 thought the picture of Kyle was scary
• (+1) Tester #1 felt the site was consistent with Fedora branding and recognizable as an official Fedora site.
• (+1) Tester #2 felt the site was consistent with Fedora branding and recognizable as an official Fedora site.
• (+1) Tester #2 felt the site was related to package maintenance.
• (-1) Tester #2 wasn't sure where to start or why he was on the site.

User Profiles

• (-1) Tester #1 was a maintainer (not owner) of one package - sssd. He could not see sssd in his list of packages because he is just a maintainer. This would cause him issues using the site.
• (-1) Tester #1 had a group he should have been a member of missing from his group memberships list on his profile.
• (-1) Tester #1 at another point in his profile saw that sssd was listed as being owned rather than simply maintained by him. So there's an inconsistency somewhere between ownership and maintainership.
• (-1) Tester #1 experienced incredibly slow loading time for Luke Macken's profile. He clicks on it at 16:51 in the video, and it fails to load up until 18:00 in the video (1 minute 9 seconds wait!). He refreshes and it then loads very quickly.
• (+1) [[FedoraCommunity/UsabilityTestingRound1/UsabilityTester2 | Tester #2]'s package listings and group memberships were correct.
• (-1) [[FedoraCommunity/UsabilityTestingRound1/UsabilityTester2 | Tester #2] owns 8 packages, only saw 5 in the sidebar with a link to 'view more.' He would have liked maybe an in-place AJAX expander so he didn't have to load a new page to see all 8.
• (+1) Tester #2 was able to find out how many packages lmacken owns and whether or not there were unpushed updates for them.
• (+1) Tester #2 found the listing of IRC nicks very helpful.

Sidebar Widgets

• (-1) [[FedoraCommunity/UsabilityTestingRound1/UsabilityTester2 | Tester #2] didn't have any interest in the 'all packages' widget in the sidebar - "Maybe Jesse would care about that. I don't."

Search

• (-1) Tester #1 could not figure out Dennis Gilmore's FAS name using people search. He could not sort by first name or last name. He simply could not complete the task using the tool.
• (+1) Tester #1 very much liked that he could search for a package and see its subpackages as well.
• (-1) Tester #1, while searching for Luke, experienced noticeably slow search time. His search started at 16:35 in the video, and he didn't receive results until 16:51 in the video (16 second delay)
• (-1) Tester #1 experienced a very slow search time for 'ipa-server'. He starts his search at 23:23. The search continues to load up until 23:07 (44 sec!), at which point he gives up and hits the back button. Firefox appears to lock up slightly. This sucks as well, it goes on to reload the page he had been on previously (sssd package details), the page takes from 23:07-24:02 in the video to reload (55 sec!)
• (-1) Tester #2 made several attempts to find Dennis Gilmore's FAS name using search with no luck.
• (-1) Tester #2 found pagination on the bottom of search results page very difficult to see.
• (0) Tester #2 for package results, would have liked to see the maintainer's name listed by the package in the results page - would asve him a click and a page load since that is what he's usually looking for.
• (-1) Tester #2 misspelled yelp while looking for it, and the search return nothing. It is very unforgiving for misspelling.
• (+1) Tester #2 was easily able to lmacken via search.
• (-1) Tester #2 clicked on search in the lower left and for the first time saw the search guide. Whoops. First instinct was to use searchbox at the top which means otherwise he'd never have seen those instructions.

People Directory

• (-1) Tester #1 took one look at it and didn't even attempt to use it, he jumped straight to search.
• (-1) Tester #2 could not find Dennis Gilmore using the people directory - difficult and slow to browse.

Package Details

• (-1) Tester #1 would have liked to see a listing of what packages a build of the package would kick off rebuilds for based on dependencies.
• (-1) Tester #1 could not find a link to Fedora 9 releases for a package because it is a now-retired release. there was no link to an archive or a way to view the 'active releases' table across ALL fedora releases.
• (0) Tester #1 noted maintainers list for yelp is quite long.
• (-1) Tester #1 experienced a slow loading time for the bugs tab of package details. He clicks on it at 19:56 in the video, and it doesn't fully load until 20:20 (24 seconds!)
• (+1) Tester #1 found it very easy to access ruby's changelog.
• (-1) Tester #2 could not find Fedora 9 (retired release) information on inkscape active releases widget.
• (-1) Tester #2 did not know that the 'builds' link with the koji icon in the center of the page linked out externally to koji - he would have retreated there to find Fedora 9 release version information for inkscape there since it wasn't in active releases. Also, there was no link to koji in the builds pages for the package.
• (-1) Tester #2 the updates page for the package didn't have a link to that package's details in bodhi.
• (-1) Tester #2 the SRPM is not on the 'sources' tab of the package. An obvious place to look for it.
• (+1) Tester #2 was easily able to find maintainer's name right on the front page of package details.
• (+1) Tester #2 was able to the bugs dashboard for NetworkManager on first try.
• (-1) Tester #2 found package bugs tab very slow to load (but understands that BZ is slow in general)
• (-1) Tester #2 did not notice the changelog menu item, and clicked over to the spec to view it rather than see the nicely formatted one. Changelog is under maintenance tools - perhaps it belongs in details?
• (-1) Tester #2 clicked on updates in main content area to go offsite to bodhi - did not notice the 'updates' link in the right sidebar that keeps you inside fedora community

Package Maintenance

• (-1) Tester #1 was unable to find 'unpushed updates' for packages he owned, and ended up searching for individual packages to get the information.
• (-1) Tester #1, when navigating from a package details page, had a bit of trouble finding a list of all the builds of packages related to him.
• (-1) Tester #1 wasn't sure what to do in order to check up on someone else's packages. He would have liked to see some kind of alerts system, email based maybe, to let him know what needs attention.
• (-1) Tester #1 wanted to be able to kick off package builds and didn't see a way how.
• (-1) Tester #1 also wanted to be able to kick off rebuilds, eg if the deps bumped because of an .so file change.
• (-1) Tester #1 wanted to be able to view a dependency tree for a given package.
• (+1) Tester #2 was able to find unpushed updates via 'my pending updates' in the right sidebar of his profile.
• (+1) Tester #2 was able to find a list of all his builds by looking at the right bar of his profile and clicking on builds - overview.
• (0) Tester #2 was unsure if the 'in-progress builds' widgets/page would automatically update or if he'd have to refresh the page to get updated information on those builds.

Navigation

• (-1) Tester #1 ended up getting confused when he was looking at a package details page, clicked on 'package maintenance', and was brought to the package details overview page rather than the main / all-packages overview page.

Overall Site Functionality

• (0) Tester #1 noted the site seemed more about monitoring than *doing*.
• (+1) Tester #2 likes how the site keeps all the information in one spot rather than across different systems (koji, bodhi, pkgdb)
• (+1) Tester #2 likes how in Fedora Community, if you're not sure how to do something the 'Fedora Community way' it's very easy to revert back to the old way of doing things with the links to pkgdb, koji, bodhi, etc
• (-1) Tester #2 thinks the fonts are a little small
• (-1) Tester #2 would like the text to be easier to scan.
• (+1) Tester #2 likes that Fedora Community is not another password/login - uses the same account he's used to using for Fedora.

Remarks By Implication

The Good

• (+1) Tester #1 found it very easy to access ruby's changelog.
• (+1) Tester #1 very much liked that he could search for a package and see its subpackages as well.
• (+1) Tester #1 felt the site was consistent with Fedora branding and recognizable as an official Fedora site.

The Bad

• (-1) Tester #1 was unable to find 'unpushed updates' for packages he owned, and ended up searching for individual packages to get the information.
• (-1) Tester #1, when navigating from a package details page, had a bit of trouble finding a list of all the builds of packages related to him.
• (-1) Tester #1 wasn't sure what to do in order to check up on someone else's packages. He would have liked to see some kind of alerts system, email based maybe, to let him know what needs attention.
• (-1) Tester #1 wanted to be able to kick off package builds and didn't see a way how.
• (-1) Tester #1 also wanted to be able to kick off rebuilds, eg if the deps bumped because of an .so file change.
• (-1) Tester #1 wanted to be able to view a dependency tree for a given package.

• (-1) Tester #1 ended up getting confused when he was looking at a package details page, clicked on 'package maintenance', and was brought to the package details overview page rather than the main / all-packages overview page.
• (-1) Tester #1 experienced a slow loading time for the bugs tab of package details. He clicks on it at 19:56 in the video, and it doesn't fully load until 20:20 (24 seconds!)
• (-1) Tester #1 would have liked to see a listing of what packages a build of the package would kick off rebuilds for based on dependencies.
• (-1) Tester #1 could not find a link to Fedora 9 releases for a package because it is a now-retired release. there was no link to an archive or a way to view the 'active releases' table across ALL fedora releases.
• (-1) Tester #1 took one look at it and didn't even attempt to use it, he jumped straight to search.
• (-1) Tester #1, while searching for Luke, experienced noticeably slow search time. His search started at 16:35 in the video, and he didn't receive results until 16:51 in the video (16 second delay)
• (-1) Tester #1 experienced a very slow search time for 'ipa-server'. He starts his search at 23:23. The search continues to load up until 23:07 (44 sec!), at which point he gives up and hits the back button. Firefox appears to lock up slightly. This sucks as well, it goes on to reload the page he had been on previously (sssd package details), the page takes from 23:07-24:02 in the video to reload (55 sec!)
• (-1) Tester #1 could not figure out Dennis Gilmore's FAS name using people search. He could not sort by first name or last name. He simply could not complete the task using the tool.
• (-1) Tester #1 was a maintainer (not owner) of one package - sssd. He could not see sssd in his list of packages because he is just a maintainer. This would cause him issues using the site.
• (-1) Tester #1 had a group he should have been a member of missing from his group memberships list on his profile.
• (-1) Tester #1 at another point in his profile saw that sssd was listed as being owned rather than simply maintained by him. So there's an inconsistency somewhere between ownership and maintainership.
• (-1) Tester #1 experienced incredibly slow loading time for Luke Macken's profile. He clicks on it at 16:51 in the video, and it fails to load up until 18:00 in the video (1 minute 9 seconds wait!). He refreshes and it then loads very quickly.
• (-1) Tester #1 thought the picture of Kyle was a little off-putting :)
• (-1) Tester #1, a new packager, had never heard of Fedora Community before the call for usability testers.

The Meh

• (0) Tester #1 noted the site seemed more about monitoring than *doing*.
• (0) Tester #1 noted maintainers list for yelp is quite long.