From Fedora Project Wiki
There are a lot of people who spend their time on QA-related activities. Some of them are Red Hat employees who are paid for this, some of them are Red Hat employees who do it in their free time, and large portion of them is the community. We would like to recognize their work as valuable and publicly thank them for their effort regularly.
Recommendations
- Name people explicitly
Don't post just a general un-targeted letter, but try to name the most helpful people explicitly. Seeing your name publicly is very gratifying and motivating. It's the least we can do for the people helping us out.
- Attach result values, but carefully
If there are any numbers connected with the names (number of executed test cases, reported bugs, etc), you can add them too and create some form of ladder. This should be done carefully, though. It will certainly please people at the top and can motivate others to increase their effort next time. But, on the other hand, it might discourage some. The results should be taken lightly and with humor in the adjacent text, rather than considering it a serious metric. You should mention that these numbers should be taken with a pinch of salt, because they are inherently incomplete (e.g. different test cases have different complexity, and also it covers just one particular area of QA activities). See the footnotes in attached examples. Adam is putting it very nicely here.
- Don't filter nor brand the contributor list
Filtering the list based on community/Red Hat membership is not recommended. You never know who participated in his/her free time and who was paid for it. Relevant discussion is in this thread. Branding Red Hat employees is not recommended as well, it might be seen as some 'elite' club. But it is recommended to acknowledge in your text that many of the contributors are Red Hat employees and specifically call out community contributors who did an outstanding job and thank them once again.
- Do not re-use old letters too much
If you re-use your letter over and over again, just changing the list of contributors, it will appear as computer-generated. That does not convey the right message. It must include a human touch if it is to be sincere. Of course there's nothing wrong with re-using some old headers and footers of the letter, but the main text should not be copy-and-pasted, but written completely anew.
- Encourage people to join the testing effort
Include information for newcomers how to join the effort and help us out in future activities.
- Cover multiple channels
When sending out the statistics/thank-you letter, it's good to send it at least to Fedora Planet, test list and Fedora community on G+.
Our regular recognition letters
Below is a list of recognition letters we try to send out regularly and their short description.
The tools used for data gathering are available in fedora-qa git:
$ git clone https://git.fedorahosted.org/git/fedora-qa.git
and wikitcms, relval and testdays repository:
$ sudo curl 'https://www.happyassassin.net/wikitcms/repo/wikitcms.repo' -o /etc/yum.repos.d/wikitcms.repo $ sudo dnf install relval testdays
Always try to gather the data as soon as possible after the relevant milestone. Due to some technical restrictions the numbers can change slightly in time; they are most accurate directly after the milestone is over.
Some tools require that you specify dates (the beginning and the end of the relevant period). For Alpha, use Branch date and Alpha Go decision date. For Beta, use Alpha Go +1 day and Beta Go date. For Final, use Beta Go +1 day and Final Go date.
Fedora Branched milestone statistics
Wiki matrices
- What: A list of all contributors who filled in some test case results in our release validation wiki matrices.
- How: Use relval:
$ relval user-stats --release 42 --milestone Alpha > stats-wiki.html
- When: Created and sent out for every Fedora Branched milestone (Alpha/Beta/Final) separately (i.e. don't include previous milestones numbers in the current stats).
Bugzilla
- What: A list of all reporters who created new bug reports in Bugzilla against a Fedora Branched release.
- How: Use stats-bugzilla.py:
$ ./stats-bugzilla.py --threshold 5 18 Final 2012-11-29 2013-01-15 > stats-bugzilla.html
The dates denote the start and end date for the specific milestone (in this example F18 Beta release and F18 Final release dates). - When: Created and sent out for every Fedora Branched milestone (Alpha/Beta/Final) separately (i.e. don't include previous milestones numbers in the current stats).
Bodhi
- What: A list of all contributors who provided feedback in Bodhi for Fedora Branched updates.
- How: Use stats-bodhi.py:
$ ./stats-bodhi.py --release F19 --startdate 2013-03-12 --enddate 2013-04-23 --username yourname --threshold 3 > stats-bodhi.html
The dates denote the start and end date for the specific milestone (in this example F19 Branch time and F19 Alpha release dates). - When: Created and sent out for every Fedora Branched milestone (Alpha/Beta/Final) separately (i.e. don't include previous milestones numbers in the current stats).
Test Days
- What: A list of the top testers at Test Day events during the release cycle. The list includes the top ten testers by number of tests run, plus any of the top ten testers by bugs referred who: a) referred at least two bugs, and b) were not already in the top ten by number of tests run.
- How: Use testdays:
testdays stats -r 22to generate statistics for Fedora 22 Test Days. - When: After all Test Days for the cycle have been run, or after the release as one of the Heroes of Fedora posts.
Quarterly statistics
Bodhi
- What: A list of all contributors who provided feedback in Bodhi for all Fedora releases in a given quarter.
- How: Use stats-bodhi.py:
$ ./stats-bodhi.py --startdate 2013-01-01 --enddate 2013-03-31 --username yourname --threshold 5 > stats-bodhi.html
The dates denote the start and end date for the specific quarter. - When: Created and sent out for every quarter separately (i.e. don't include previous quarters numbers in the current stats).