2016-10-24 Cloud

Fedora Test Day

Fedora Cloud Test day

Date	2016-10-24
Time	all day

Website	QA/Test Days
IRC	#fedora-test-day (webirc)
Mailing list	test

Can't make the date?
If you come to this page before or after the test day is completed, your testing is still valuable, and you can use the information on this page to test, file any bugs you find at Bugzilla, and add your results to the results section. If this page is more than a month old when you arrive here, please check the current schedule and see if a similar but more recent Test Day is planned or has already happened.

What to test?[edit]

Today's installment of Fedora Test Day will focus on Cloud, Atomic Host images for Fedora 25.

Who's available[edit]

The following cast of characters will be available testing, workarounds, bug fixes, and general discussion ...

Development -
Quality Assurance -
Community Members - dustymabe, jasonbrooks, kushal, bowlofeggs

Prerequisite for Test Day[edit]

You will require at least a system with libvirt/kvm installed. Having access to any Openstack or other Cloud IAAS system where you can upload an image will be even better.

Download the images from below links

Vagrant libvirt Cloud Base

List of AMIs

Image Name	AMI id	Destination	Virt type	Vol type
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (us-west-2)	ami-2d4bef4d	hvm	gp2
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (us-west-2)	ami-7d48ec1d	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (us-west-1)	ami-c4450ea4	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (sa-east-1)	ami-1bd64a77	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (eu-west-1)	ami-1b327b68	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (eu-central-1)	ami-2645bc49	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (ap-southeast-2)	ami-100b3673	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (ap-southeast-1)	ami-afe948cc	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (ap-northeast-1)	ami-5abd1c3b	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (us-west-1)	ami-6a400b0a	hvm	gp2
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (sa-east-1)	ami-39d64a55	hvm	gp2
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (eu-west-1)	ami-52337a21	hvm	gp2
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (eu-central-1)	ami-9d44bdf2	hvm	gp2
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (ap-southeast-2)	ami-7a0a3719	hvm	gp2
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (ap-southeast-1)	ami-71e94812	hvm	gp2
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (ap-northeast-1)	ami-8fba1bee	hvm	gp2
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (us-east-1)	ami-a7461bb0	hvm	standard
Fedora-Atomic-25-20161023.n.0.x86_64	EC2 (us-east-1)	ami-71471a66	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-west-2)	ami-a74befc7	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-west-2)	ami-a54befc5	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-west-1)	ami-bc470cdc	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (sa-east-1)	ami-e2da468e	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (eu-west-1)	ami-05337a76	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (eu-central-1)	ami-2b45bc44	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-southeast-2)	ami-33053850	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-southeast-1)	ami-9be746f8	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-west-1)	ami-07460d67	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (sa-east-1)	ami-47d4482b	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (eu-west-1)	ami-51347d22	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (eu-central-1)	ami-7db84012	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-southeast-2)	ami-dc0a37bf	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-southeast-1)	ami-99e746fa	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-northeast-1)	ami-ffbf1e9e	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-northeast-1)	ami-37802156	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-east-1)	ami-ed401dfa	paravirtual	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-east-1)	ami-f1461be6	paravirtual	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-west-2)	ami-194feb79	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (sa-east-1)	ami-ce069ba2	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-west-1)	ami-a1470cc1	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (eu-west-1)	ami-91337ae2	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (eu-central-1)	ami-fb46bf94	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-southeast-2)	ami-130b3670	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-southeast-1)	ami-a4e849c7	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-west-2)	ami-114aee71	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-northeast-1)	ami-3cbe1f5d	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (sa-east-1)	ami-cd069ba1	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-west-1)	ami-c6450ea6	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (eu-west-1)	ami-b1327bc2	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (eu-central-1)	ami-f546bf9a	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-southeast-2)	ami-1a0b3679	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-southeast-1)	ami-2ce8494f	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (ap-northeast-1)	ami-64812005	hvm	gp2
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-east-1)	ami-d04a17c7	hvm	standard
Fedora-Cloud-Base-25-20161023.n.0.x86_64	EC2 (us-east-1)	ami-cf4419d8	hvm	gp2

Here's a chunk which is commonly used for most Test Days. Replace XX with whatever Fedora release is pending:

An updated Fedora 25 nightly Cloud/Atomic Host image

How to test?[edit]

Boot the images on a cloud system, or on Vagrant or using testcloud on a local system.
Execute the tests.

Note: For the service manipulation tests we are stopping/disabling chronyd service on Fedora 25 systems.

Known Issues[edit]

For the Atomic qcow2 image we have a known docker storage issue. Use the following cloud-config to start an instance in a cloud.

   #cloud-config
   bootcmd:
   - sed s/After=cloud-final.service/After=cloud-init-local.service/ /usr/lib/systemd/system/docker-storage-setup.service > /etc/systemd/system/docker-storage-setup.service
   - systemctl daemon-reload
   - systemctl start --no-block docker.service

Run the tests[edit]

Visit the result page and click on the column title links to see the tests that need to be run: most column titles are links to a specific test case. Follow the instructions there, then enter your results by clicking the Enter result button for the test.

Exploratory testing[edit]

If you run out of test cases, congratulations! But that's not the end! You can still help out by playing around with the tool in whatever ways you can think of: try out all the things you can find, and try writing different Fedora images with it. If you have systems with different Windows versions, try running the tool on all of them. Get creative! Any problems you find please file a bug, or report to the IRC channel.

Reporting bugs[edit]

If you have problems with any of the tests, report a bug to Bugzilla. If you are unsure about exactly how to file the report or what other information to include, just ask on IRC and we will help you.

IRC Channel name: #fedora-cloud, #fedora-test-day
Server: Freenode

Test Results[edit]

Results transferred from the result page 2018-02-06.

Basic Tests[edit]

User	Profile	Base startup	System logging	Services start	SELinux	Service manipulation	References
bowlofeggs	Vagrant libvirt Cloud Base	pass	pass	pass	pass	pass
jasonbrooks	qcow2 -- atomic -- libvirt	pass	pass	pass ^[1]	pass	pass ^[2]	↑ `systemctl --all --failed` reported no failed services, despite https://bugzilla.redhat.com/show_bug.cgi?id=1387934 ↑ Works, despite continual, maddening "service changed on disk. Run 'systemctl daemon-reload' to reload" bug: https://bugzilla.redhat.com/show_bug.cgi?id=1384150
jberkus	AWS, us-west-2, Fedora-Atomic-25-20161023.n.0.x86_64-us-west-2-HVM-gp2-0 (ami-2d4bef4d), T2 Small	pass ^[1]	pass	pass	pass	warn ^[2] pass	↑ Booted on AWS ↑ Got bogus "Warning: docker.service changed on disk. Run 'systemctl daemon-reload' to reload units." on status.
kushal		pass	pass	pass ^[1]	pass	pass pass	↑ * With the userdata patch from Dusty
kushal	AWS t2.medium us-west-1 ami-c4450ea4	pass	pass	pass	pass	pass
kushal	Openstack	pass	pass	pass	pass	pass
trishnag			pass
trishnag	Openstack		pass	pass	pass	pass

Atomic Tests[edit]

User	Profile	Image boots	Upgrade works	Rollback works	References
dustymabe	openstack newton	pass
dustymabe	openstack newton atomic qcow	pass
jasonbrooks	qcow2 -- atomic -- libvirt	pass	pass	pass
jasonbrooks	vagrant -- atomic -- libvirt	pass	pass ^[1]		↑ I tested upgrade by first deploying a previous version: `atomic host deploy 25.32 && reboot`
jberkus	AWS, us-west-2, Fedora-Atomic-25-20161023.n.0.x86_64-us-west-2-HVM-gp2-0 (ami-2d4bef4d), T2 Small	pass	pass ^[1]		↑ Unable to upgrade because I installed the latest version. However, "rpm-ostree upgrade" gave the correct result.
kushal	AWS t2.medium us-west-1 ami-c4450ea4	pass
kushal	Openstack	pass
trishnag	Openstack	pass	pass	pass

Docker tests[edit]

User	Profile	Docker install	Docker storage setup	Docker daemon	References
dustymabe	openstack newton atomic qcow	pass	fail ^[1]	pass ^[2]	↑ RHBZ #1387934 Had to work around the bug with user data from https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/message/R7S4AMKFAZZBDROBS4D76PRSOK7OIW6O/ ↑ passed with workaround from #1387934
jasonbrooks	qcow2 -- atomic -- libvirt		fail ^[1]	warn ^[2]	↑ RHBZ #1387934 Didn't run automatically, so docker started w/ loopback storage. I worked around w: `systemctl stop docker && rm -rf /var/lib/docker && docker-storage-setup && systemctl start docker` ↑ Started automatically, but w/ loopback storage, worked around by running docker-storage-setup manually.
jberkus	AWS, us-west-2, Fedora-Atomic-25-20161023.n.0.x86_64-us-west-2-HVM-gp2-0 (ami-2d4bef4d), T2 Small	pass ^[1]	fail ^[2]	pass ^[3]	↑ Docker 1.12 installed. ↑ docker-storage-setup.service: Job docker-storage-setup.service/start deleted to break ordering cycle starting with multi Starting Docker Storage Setup... CHANGED: partition=2 start=616448 old: size=11966464 end=12582912 new: size=20355072,end=20971520 Physical volume "/dev/xvda2" changed 1 physical volume(s) resized / 0 physical volume(s) not resized WARNING: D-Bus notification failed: The name com.redhat.lvmdbus1 was not provided by any .service files ERROR: Docker has been previously configured for use with devicemapper graph driver. Not creating a new thin pool as exi INFO: Docker state can be reset by stopping docker and by removing /var/lib/docker directory. This will destroy existing docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE Failed to start Docker Storage Setup. docker-storage-setup.service: Unit entered failed state. docker-storage-setup.service: Failed with result 'exit-code'. Also, Docker Pool reports that it's using a loopback which is 100GB in size. This VM only has 11GB RAM + Storage total. This seems like an issue. ↑ Started with loopback storage; had to be manually torn down and docker-storage-setup rerun.
lnie	openstack	pass	fail ^[1]		↑ RHBZ #1388000

More Atomic related tests[edit]

User	Profile	Atomic command	Atomic read only mounts	Atomic root mount	Atomic tmp mount	Atomic tmp writable	Atomic journal written
dustymabe	openstack newton atomic qcow	pass	pass	pass	pass	pass	pass
jasonbrooks	qcow2 -- atomic -- libvirt	pass	pass	pass	pass	pass	pass
jberkus	AWS, us-west-2, Fedora-Atomic-25-20161023.n.0.x86_64-us-west-2-HVM-gp2-0 (ami-2d4bef4d), T2 Small	pass	pass	pass	pass	pass	pass
kushal	AWS t2.medium us-west-1 ami-c4450ea4	pass	pass	pass	pass	pass	pass
kushal	Openstack	pass	pass	pass	pass	pass	pass

Vagrant test[edit]

User	Profile	eth0 naming	References
bowlofeggs	Vagrant libvirt Cloud Base	pass
jasonbrooks	vagrant -- atomic -- libvirt	pass

[1] systemctl --all --failed reported no failed services, despite https://bugzilla.redhat.com/show_bug.cgi?id=1387934

[2] Works, despite continual, maddening "service changed on disk. Run 'systemctl daemon-reload' to reload" bug: https://bugzilla.redhat.com/show_bug.cgi?id=1384150

[3] Booted on AWS

[4] Got bogus "Warning: docker.service changed on disk. Run 'systemctl daemon-reload' to reload units." on status.

[5] * With the userdata patch from Dusty

[6] I tested upgrade by first deploying a previous version: atomic host deploy 25.32 && reboot

[7] Unable to upgrade because I installed the latest version. However, "rpm-ostree upgrade" gave the correct result.

[8] RHBZ #1387934 Had to work around the bug with user data from https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/message/R7S4AMKFAZZBDROBS4D76PRSOK7OIW6O/

[9] ssed with workaround from #1387934

[10] RHBZ #1387934 Didn't run automatically, so docker started w/ loopback storage. I worked around w: systemctl stop docker && rm -rf /var/lib/docker && docker-storage-setup && systemctl start docker

[11] Started automatically, but w/ loopback storage, worked around by running docker-storage-setup manually.

[12] Docker 1.12 installed.

[13] r-storage-setup.service: Job docker-storage-setup.service/start deleted to break ordering cycle starting with multi Starting Docker Storage Setup... CHANGED: partition=2 start=616448 old: size=11966464 end=12582912 new: size=20355072,end=20971520 Physical volume "/dev/xvda2" changed 1 physical volume(s) resized / 0 physical volume(s) not resized WARNING: D-Bus notification failed: The name com.redhat.lvmdbus1 was not provided by any .service files ERROR: Docker has been previously configured for use with devicemapper graph driver. Not creating a new thin pool as exi INFO: Docker state can be reset by stopping docker and by removing /var/lib/docker directory. This will destroy existing docker-storage-setup.service: Main process exited, code=exited, status=1/FAILURE Failed to start Docker Storage Setup. docker-storage-setup.service: Unit entered failed state. docker-storage-setup.service: Failed with result 'exit-code'. Also, Docker Pool reports that it's using a loopback which is 100GB in size. This VM only has 11GB RAM + Storage *total*. This seems like an issue.

[14] Started with loopback storage; had to be manually torn down and docker-storage-setup rerun.

[15] RHBZ #1388000

[1]

[2]

[1]

[2]

[1]

[1]

[1]

[1]

[2]

[1]

[2]

[1]

[2]

[3]

[1]

Search

Test Day:2016-10-24 Cloud

Contents