From Fedora Project Wiki

No edit summary
(Drop direct Test Days category membership)
 
(62 intermediate revisions by 7 users not shown)
Line 56: Line 56:
=== Planned Tests ===
=== Planned Tests ===


* [[QA:Testcase_thermostat_in_path]]
* [[QA:Testcase_thermostat_storage]]
* [[QA:Testcase_thermostat_storage]]
* [[QA:Testcase_thermostat_logging]]
* [[QA:Testcase_thermostat_logging]]
Line 69: Line 70:
* [[QA:Testcase_thermostat_web_basic]]
* [[QA:Testcase_thermostat_web_basic]]
* [[QA:Testcase_thermostat_web_auth]]
* [[QA:Testcase_thermostat_web_auth]]
* [[QA:Testcase_thermostat_web_tls]]
* [[QA:Testcase_thermostat_remote_monitoring]]
* [[QA:Testcase_thermostat_remote_monitoring]]


Line 78: Line 78:
== Test Results ==
== Test Results ==


If you have problems with any of the tests, try and report a bug. Thermostat bugs should be reported to [https://icedtea.classpath.org/bugzilla/enter_bug.cgi?product=Thermostat Thermostat's Bugzilla]. You will need an icedtea bugzilla account to report bugs for Thermostat. Creating one is easy, and we will help you do this if you ask in [irc://irc.freenode.net/#fedora-test-day IRC].  
If you have problems with any of the tests, try and report a bug. Thermostat bugs should be reported to [http://icedtea.classpath.org/bugzilla/enter_bug.cgi?product=Thermostat Thermostat's Bugzilla]. You will need an icedtea bugzilla account to report bugs for Thermostat. Creating one is easy, and we will help you do this if you ask in [irc://irc.freenode.net/#fedora-test-day IRC].  


If you are not sure of the appropriate component, please check in [irc://irc.freenode.net/#fedora-test-day IRC] before filing, there are many possibilities. If you are unsure about exactly how to file the report or what other information to include, just ask on [irc://irc.freenode.net/#fedora-test-day IRC] and we will help you.  
If you are not sure of the appropriate component, please check in [irc://irc.freenode.net/#fedora-test-day IRC] before filing, there are many possibilities. If you are unsure about exactly how to file the report or what other information to include, just ask on [irc://irc.freenode.net/#fedora-test-day IRC] and we will help you.  
Line 86: Line 86:
{|
{|
! User
! User
! [[QA:Testcase_thermostat_upgrade|Upgrade]]
![[QA:Testcase_thermostat_in_path|Version]]
! [[QA:Testcase_thermostat_service|Service]]
![[QA:Testcase_thermostat_storage|Storage]]
! [[QA:Testcase_thermostat_gui|GUI]]
![[QA:Testcase_thermostat_logging|Logging]]
! [[QA:Testcase_thermostat_shell|Shell]]
![[QA:Testcase_thermostat_agent|Agent]]
! [[QA:Testcase_thermostat_shell_other|Commands]]
![[QA:Testcase_thermostat_service|Service]]
! [[QA:Testcase_thermostat_web|Web Service]]
![[QA:Testcase_thermostat_swing_client|Swing Client]]
! [[QA:Testcase_thermostat_web_remote_with_security|Remote Web with Security]]
![[QA:Testcase_thermostat_shell|Shell]]
! [[QA:Testcase_thermostat_remote_mongodb|Remote Mongodb]]
![[QA:Testcase_thermostat_command_channel|Cmd Chan]]
! [[QA:Testcase_thermostat_heapdump|Heapdump]]
![[QA:Testcase_thermostat_heapdump|Heap]]
! [[QA:Testcase_thermostat_thread_monitor|Threads]]
![[QA:Testcase_thermostat_thread_monitor|Thread]]
![[QA:Testcase_thermostat_stand-alone_commands|Commands]]
![[QA:Testcase_thermostat_custom_plugin|Plug-in]]
![[QA:Testcase_thermostat_web_basic|Web Basic]]
![[QA:Testcase_thermostat_web_auth|Web Auth]]
![[QA:Testcase_thermostat_remote_monitoring|Remote]]
! References
! References
|-
|-
| [[User:SampleUser|Sample User]]
| [[User:SampleUser|Sample User]]
| {{result|none}}  
| {{result|none}}  
| {{result|none}}
| {{result|pass}}
| {{result|pass}}
| {{result|none}}
| {{result|none}}
| {{result|none}}  
| {{result|none}}  
| {{result|pass}}
| {{result|none}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|none}}
| {{result|none}}
Line 111: Line 121:
| <references/>
| <references/>
|-
|-
| [[User:lzachar|lzachar]]
| {{result|pass}} <ref> missing manpages [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1438 thermostat#1438]</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|fail}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1442 thermostat#1442] </ref>
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=994 thermostat#994]</ref> <ref>[http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1444 thermostat#1444]</ref>
| {{result|none}}
| {{result|pass}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:jerboaa|Severin Gehwolf]]
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|fail}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1442 thermostat#1442] </ref> See also: <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1448 thermostat#1448] </ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|fail}} Encountered <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1446 thermostat#1443]</ref> and <ref>[http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1447 thermostat#1447]</ref>
| <references/>
|-
| [[User:aazores|Andrew Azores]]
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|fail}} <ref>Build failure on live image [http://paste.fedoraproject.org/14054/30067136/ paste.fedoraproject.org]</ref>
| {{result|warn}} <ref group="long">Thermostat gui failed to start on one occasion but worked immediately after. [http://paste.fedoraproject.org/14051/36932954/ paste.fedoraproject.org]. "thermostat agent -d" also was not producing error output when unable to connect to tomcat, even with logging set up as described in earlier test case, and with error messages being produced and visible in tomcat logs.</ref>
| {{result|pass}}
| {{result|none}}
| <references/>
|-
| [[User:ptisnovs|Pavel Tišnovský]]
| {{result|pass}}
| {{result|fail}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1449 thermostat#1449]</ref> enh: <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1445 thermostat#1445]</ref>
| {{result|pass}}
| {{result|warn}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1441 thermostat#1441]</ref>
| {{result|pass}}
| {{result|warn}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1439 thermostat#1439]</ref> <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1440 thermostat#1440]</ref>
| {{result|none}}
| {{result|none}}
| {{result|pass}}
| {{result|pass}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:omajid|omajid]]
| {{result|pass}} <ref>Test passes, but also shows messages about creating files and directories on first run </ref>
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} <ref> Test passes, but not very obvious [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1452 icedtea#1452]</ref>
| {{result|warn}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1459 thermostat#1459]</ref> <ref>[http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1457 thermostat#1457]</ref> <ref>The 'search' icon in the test case screenshot is blank when I run thermostat </ref>
| {{result|fail}} <ref>Crash on exit [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1442 icedtea#1442]</ref> <ref>Error on repeated list-vm/disconnect calls [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1453 icedtea#1453]</ref>
| {{result|pass}}
| {{result|fail}} <ref group="long">When I double click on a heap dump, the live VM starts using 100% cpu and becomes unresponsive. After ~5 minutes, the OOM killer kills the thermostat swing client</ref>
| {{result|warn}} <ref>pausing/resuming thread monitoring 'fills' in unknown data for the thread timeline (how does it get this data?).</ref>
| {{result|pass}}
| {{result|warn}} <ref>I had to 'yum install mercurial maven-local fusesource-pom' to be able to build</ref> <ref>Plugins can only be installed as root</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|none}}
| <references/>
|-
| [[User:jfabriko|Jana Fabrikova]]
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|fail}}
<ref>[http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1450 thermostat#1450] dump-heap header more info wanted</ref>
<ref group="long">
Generally in the shell it was not very intuitive which ids to use for which
commands. It would be nice to have a list of example usages somewhere + where
can i find the ids, such as
"If you want to create a heap dump you need an id of
a _currently running_ vm, which can be found by list-vms. The typical call
in the shell with minimal number of options is
Thermostat> dump-heap -a ble123ble-very-long-id -v 4567
where -a is for specifying the host_id and -v for the vm_id."
</ref>
<ref group="long">
One more thing that surprised me in the shell was that if I run the sequence
thermostat storage --start,
thermostat agent (in another terminal),
thermostat shell,
then the connect -d ... command tries to connect to a db, whereas if I run
some other command in the shell before the connect (e.g. list-vms) then this
command autoconnects and subsequent call to connect tells me the connection
already exists. (Maybe the side effects of commands could be noted somewhere.)
</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} The red icon in "Threads" without implicitly visible label that would say that it is needed to click on it to see any results (recording/stop recording)
| {{result|none}}
| {{result|fail}} <ref group="long">on the live image i encountered a build failure:
[ERROR] Failed to execute goal on project thermostat-plug-in-storage-common: Could not resolve dependencies for project com.redhat.thermostat.tutorial.kernel.cmdline:thermostat-plug-in-storage-common:bundle:0.0.2-SNAPSHOT: Failed to collect dependencies for [org.osgi:org.osgi.core:jar:4.2.0 (provided), org.osgi:org.osgi.compendium:jar:4.2.0 (provided), com.redhat.thermostat:thermostat-storage-core:jar:0.8.0-SNAPSHOT (compile), com.redhat.thermostat:thermostat-common-core:jar:0.8.0-SNAPSHOT (compile)]: Failed to read artifact descriptor for org.fusesource.jansi:jansi:jar:1.9: The repository system is offline but the artifact org.fusesource:fusesource-pom:pom:1.8 is not available in the local repository. -> [Help 1]</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:apetrus|Andriy Petrus]]
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} <ref> Test passed, but "starting agent now..." message is not very clear whether it has fully started or not</ref>
| {{result|pass}}
| {{result|fail}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1442 thermostat#1442]</ref>
| {{result|pass}}
| {{result|fail}} <ref group="long"> Occasionally, after right clicking on the graph and then clicking "Heap Dump" more than once, the VM becomes unresponsive. Not 100% reproducible</ref>
| {{result|warn}} <ref group="long"> It is not very intuitive/obvious for average user that this feature starts working/displaying threads only after the red "Start Recording" is clicked</ref>
| {{result|pass}}
| {{result|fail}} <ref> Build failed on F19 live image [http://paste.fedoraproject.org/14093/93350241 paste.fedoraproject.org]</ref>
| {{result|none}}
| {{result|none}}
| {{result|none}}
| <references/>
|-
| [[User:obryan|Giovanni Astarita]]
| {{result|pass}}
| {{result|pass}} <ref> Test passed, but with unplanned tests I got a bug [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1451 thermostat#1451]</ref> <ref> Test passed, but with unplanned tests I got a bug [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1454 thermostat#1454]</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1452 thermostat#1452]</ref>
| {{result|pass}}
| {{result|fail}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1442 thermostat#1442]</ref>
| {{result|pass}}
| {{result|pass}}
| {{result|warn}} <ref> Not so obvious for average user. Need label for red button.</ref>
| {{result|pass}}
| {{result|fail}} <ref> Build failed on testday F19 live image [http://fpaste.org/14307/07964136/ fpaste.org]</ref>
| {{result|fail}} <ref> [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1456 thermostat#1456] </ref>
| {{result|fail}} <ref> Failed probably due to the previous test [http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=1456 thermostat#1456] </ref>
| {{result|none}}
| <references/>
|}
|}


[[Category:Test Days]] [[Category:Fedora 19 Test Days]]
== Long comments ==
<references group="long" />
 
[[Category:Fedora 19 Test Days]]

Latest revision as of 19:29, 26 June 2015

Fedora Test Days
[Thermostat 0.9.0]

Date 2013-05-23
Time all day

Website Thermostat Website
IRC #fedora-test-day (webirc)
Mailing list test


Can't make the date?
If you come to this page before or after the test day is completed, your testing is still valuable, and you can use the information on this page to test, file any bugs you find at our bug tracker, and add your results to the results section. If this page is more than a month old when you arrive here, please check the current schedule and see if a similar but more recent Test Day is planned or has already happened.

What to Test?[edit]

Today's installment of Fedora Test Day will focus on Thermostat

Yay, Thermostat 0.9.0 is out!
As Thermostat 0.9.0 has been released recently, today's test day can provide a nice way for you to get familiar with the new version, its looks and all the important functionality - especially if you are new to them.

Who's Available[edit]

The following cast of characters will be available testing, workarounds, bug fixes, and general discussion ...

Prerequisite for Test Day[edit]

Architecture SHA256SUM
x86_64 621148638bbf87a8fe3cdf3cc51615e4cf4d311f5c80abcdc4852229935e99f6
i686 32bb499443145799c3236396be1cddc87aba46caa259c3721f8d978df6bfd0ad
  • You will either need a physical machine or a virtual machine running a pre-release of Fedora 19
  • Make sure that your system is up-to-date. You can update your system using the update manager.
  • If you have a F19 installation, you should have the following packages installed:
thermostat-0.8.0-0.1.20130521hg97e66ed2e4ae.fc19
thermostat-webapp-0.8.0-0.1.20130521hg97e66ed2e4ae.fc19

You can download this build for your architecture from Koji and use sudo yum localinstall thermostat-0.8.0-0.1.20130521hg97e66ed2e4ae.fc19*.rpm thermostat-webapp-0.8.0-0.1.20130521hg97e66ed2e4ae.fc19*.rpm in order to install it.

Perform Testing[edit]

Please perform as many of the test cases listed as you have the time and the resources to complete, and fill out your results in the table below. You do not need a Fedora account to fill in the table. It is recommended to perform the tests in order, since some tests build on things explained in previous tests.

Preliminary Information[edit]

Note that thermostat consists of four (4) services. Each of which can be run on a different host. The first service is Thermostat storage (a.k.a. the database). The second one is the Thermostat webservice. This one has been introduced for security reasons. If you do not plan to use Thermostat in a distributed environment - or don't care about security - this service can probably be skipped. The third service is the Thermostat agent. It is a service which one needs to run on every host where JVMs should be monitored. The last component in this quartet are various forms of Thermostat clients.

Planned Tests[edit]

Unplanned Testing[edit]

As well as running the formal test cases, you can help simply by using Thermostat and/or Thermostat's Eclipse integration and report any problems you come across in the course of your typical use, even if they do not match up with any of the test cases. If unsure if something is a bug feel free to ask in #fedora-test-day.

Test Results[edit]

If you have problems with any of the tests, try and report a bug. Thermostat bugs should be reported to Thermostat's Bugzilla. You will need an icedtea bugzilla account to report bugs for Thermostat. Creating one is easy, and we will help you do this if you ask in IRC.

If you are not sure of the appropriate component, please check in IRC before filing, there are many possibilities. If you are unsure about exactly how to file the report or what other information to include, just ask on IRC and we will help you.

Once you have completed the tests, add your results to the Results table below, following the example results from the first line as a template. The first column should be your name with a link to your User page in the Wiki if you have one, and the second should be a link to the Smolt profile of the system you tested. For each test case, use the result template to describe your result, following the examples in the Sample user row.

User Version Storage Logging Agent Service Swing Client Shell Cmd Chan Heap Thread Commands Plug-in Web Basic Web Auth Remote References
Sample User
none
none
Pass pass
none
none
Pass pass
none
Pass pass
Pass pass
Pass pass
none
Pass pass
Pass pass
Warning warn
[1]
Fail fail
[2]
  1. Test pass, but also encountered icedtea#696372
  2. RHBZ #thermostat
lzachar
Pass pass
[1]
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Fail fail
[2]
Pass pass
Pass pass
Pass pass
Warning warn
[3] [4]
none
Pass pass
none
none
Severin Gehwolf
none
none
none
none
none
none
Fail fail
[1] See also: [2]
none
none
none
none
none
none
none
Fail fail
Encountered [3] and [4]
Andrew Azores
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Fail fail
[1]
Warning warn
[long 1]
Pass pass
none
  1. Build failure on live image paste.fedoraproject.org
Pavel Tišnovský
Pass pass
Fail fail
[1] enh: [2]
Pass pass
Warning warn
[3]
Pass pass
Warning warn
[4] [5]
none
none
Pass pass
Pass pass
none
none
none
none
none
omajid
Pass pass
[1]
Pass pass
Pass pass
Pass pass
Warning warn
[2]
Warning warn
[3] [4] [5]
Fail fail
[6] [7]
Pass pass
Fail fail
[long 2]
Warning warn
[8]
Pass pass
Warning warn
[9] [10]
Pass pass
Pass pass
none
  1. Test passes, but also shows messages about creating files and directories on first run
  2. Test passes, but not very obvious icedtea#1452
  3. thermostat#1459
  4. thermostat#1457
  5. The 'search' icon in the test case screenshot is blank when I run thermostat
  6. Crash on exit icedtea#1442
  7. Error on repeated list-vm/disconnect calls icedtea#1453
  8. pausing/resuming thread monitoring 'fills' in unknown data for the thread timeline (how does it get this data?).
  9. I had to 'yum install mercurial maven-local fusesource-pom' to be able to build
  10. Plugins can only be installed as root
Jana Fabrikova
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Pass pass
Fail fail

[1] [long 3] [long 4]

Pass pass
Pass pass
Warning warn
The red icon in "Threads" without implicitly visible label that would say that it is needed to click on it to see any results (recording/stop recording)
none
Fail fail
[long 5]
none
none
none
  1. thermostat#1450 dump-heap header more info wanted
Andriy Petrus
Pass pass
Pass pass
Pass pass
Pass pass
Warning warn
[1]
Pass pass
Fail fail
[2]
Pass pass
Fail fail
[long 6]
Warning warn
[long 7]
Pass pass
Fail fail
[3]
none
none
none
  1. Test passed, but "starting agent now..." message is not very clear whether it has fully started or not
  2. thermostat#1442
  3. Build failed on F19 live image paste.fedoraproject.org
Giovanni Astarita
Pass pass
Pass pass
[1] [2]
Pass pass
Pass pass
Warning warn
[3]
Pass pass
Fail fail
[4]
Pass pass
Pass pass
Warning warn
[5]
Pass pass
Fail fail
[6]
Fail fail
[7]
Fail fail
[8]
none
  1. Test passed, but with unplanned tests I got a bug thermostat#1451
  2. Test passed, but with unplanned tests I got a bug thermostat#1454
  3. thermostat#1452
  4. thermostat#1442
  5. Not so obvious for average user. Need label for red button.
  6. Build failed on testday F19 live image fpaste.org
  7. thermostat#1456
  8. Failed probably due to the previous test thermostat#1456

Long comments[edit]

  1. Thermostat gui failed to start on one occasion but worked immediately after. paste.fedoraproject.org. "thermostat agent -d" also was not producing error output when unable to connect to tomcat, even with logging set up as described in earlier test case, and with error messages being produced and visible in tomcat logs.
  2. When I double click on a heap dump, the live VM starts using 100% cpu and becomes unresponsive. After ~5 minutes, the OOM killer kills the thermostat swing client
  3. Generally in the shell it was not very intuitive which ids to use for which commands. It would be nice to have a list of example usages somewhere + where can i find the ids, such as "If you want to create a heap dump you need an id of a _currently running_ vm, which can be found by list-vms. The typical call in the shell with minimal number of options is Thermostat> dump-heap -a ble123ble-very-long-id -v 4567 where -a is for specifying the host_id and -v for the vm_id."
  4. One more thing that surprised me in the shell was that if I run the sequence thermostat storage --start, thermostat agent (in another terminal), thermostat shell, then the connect -d ... command tries to connect to a db, whereas if I run some other command in the shell before the connect (e.g. list-vms) then this command autoconnects and subsequent call to connect tells me the connection already exists. (Maybe the side effects of commands could be noted somewhere.)
  5. on the live image i encountered a build failure: [ERROR] Failed to execute goal on project thermostat-plug-in-storage-common: Could not resolve dependencies for project com.redhat.thermostat.tutorial.kernel.cmdline:thermostat-plug-in-storage-common:bundle:0.0.2-SNAPSHOT: Failed to collect dependencies for [org.osgi:org.osgi.core:jar:4.2.0 (provided), org.osgi:org.osgi.compendium:jar:4.2.0 (provided), com.redhat.thermostat:thermostat-storage-core:jar:0.8.0-SNAPSHOT (compile), com.redhat.thermostat:thermostat-common-core:jar:0.8.0-SNAPSHOT (compile)]: Failed to read artifact descriptor for org.fusesource.jansi:jansi:jar:1.9: The repository system is offline but the artifact org.fusesource:fusesource-pom:pom:1.8 is not available in the local repository. -> [Help 1]
  6. Occasionally, after right clicking on the graph and then clicking "Heap Dump" more than once, the VM becomes unresponsive. Not 100% reproducible
  7. It is not very intuitive/obvious for average user that this feature starts working/displaying threads only after the red "Start Recording" is clicked