Infrastructure private cloud

Background

Fedora Infrastructure is looking to setup a private eucalyptus cloud instance in 2012. This cloud instance will be used in a number of ways to benefit Fedora. We evaluated a number of cloud technologies and decided (at least for now) on eucalyptus as the best fit for our needs.

Why Eucalyptus

• Open Source

• Active Community

• Deployable now

• Instances can be VLAN private so they cannot interfere with each other.

Use cases

• Fedora QA may use instances with it's AutoQA setup. Instances would be created, tests run and destroyed. It's unknown how many instances we would need here.

• Infrastructure Development hosts may be moved to this cloud. These instances could possibly be 'on demand' when development needs to take place. Currently we have about 8 development instances. These may need persistent storage.

• Infrastructure Staging hosted may be moved to this cloud. Some of these may be 'always on' and some may be on demand. Currently we have about 13 of these instances. These may need persistent storage.

• Chainbuilding / Kopers may use this cloud to build chains of packages that are not yet in Fedora and thus cannot be build via scratch builds in the existing buildsystem. These may also be used for spinning test live or install images by QA. This may be open to Fedora contributors or restricted to a subset such as packagers.

• Test instances may be used for testing new tech or applications as a proof of concept before persuing a RFR. We currently have several publictest instances.

• We may want to move some of our one-off instances that are outside phx2 into the cloud for easier management. Things like keyservers, unbound instances, listservers or hosted resources.

• Mass rebuilds of Fedora packages. This could be done for testing a new global rpm/package change, or to discover FTBFS (Fails to build from source) packages. This would use as many builders as we could easily spin up to reduce time for building all 10,000+ Fedora packages. Could use the chainbuilding setup as above as a scaffolding. Additionally, extra builder instances could be potentially used by the official build system during mass rebuilds to reduce rebuild time.

Further down the road:

• Instances for qa/packagers to test new packages or track down bugs.

• Instances for demos or events to show off Fedora.

For initial deployment, we would need to be able to run ~30 or so instances at a time with ability to grow rapidly above that for qa and building needs.

Dependencies

• Need a way to easily provision new instances with limited admin intervention. Looking at ansible for this task.

• Would like to be able to create images via kickstart and normal install/deployment methods if needed.

• Hardware needs to be ordered and installed.

• Public IP addresses need to be made available.

• Would be nice to get full EPEL packages to deploy with.

Setup / deployment

This hardware will be on the 'edge' of the network and not connected to the rest of Fedora Infrastructure except via external networks. This will allow us to us external ip's and make sure the cloud instance doesn't have access to anything in the regular Fedora Infrastructure. Storage will be on the local servers for caching with additional netapp space for images and data.

Implementation overview / timelines

2012-04 - Hardware is being determined and finalized.

2012-07 - Initial hardware setup and install

2012-08 - Initial use cases setup and tested

2012-09 - Announce availability and collect more use cases.

2012-10 - Evaluate load and expansion needs.

Policies

This section is currently under discussion. We need to setup clear policies on usage and access to the private cloud. In general we plan to open things to a small group of trusted contributors, take their feedback and usage and expand access out to larger groups as capacity and desire allows.

(This section is a DRAFT)

Users or groups that need rare one off images can simply request one via a ticket. Users or groups that often need instances will be granted accounts to spin up and down their own images.

Instances may be rebooted at any time. Save your data off often.

Persistent storage may be available as seperate volumes. Data retention and Quotas may be imposed on this data.

Default instance time to live would be a week.

Default network policy will allow only ports 80, 443, and 22 tcp.

Instances are assist in furthering the work related to the Fedora Project. Please don't use them for unrelated activities.

We reserve the right to shutdown, delete or revoke access to any instances at any time for any reason.