From Fedora Project Wiki
m (→‎Requirement #4: Needs for Big Data Image: deduplication + small understandability improvement)
(→‎Product Objectives: Update the general statement)
 
(19 intermediate revisions by 7 users not shown)
Line 1: Line 1:
{{header|cloud-sig}}
{{header|cloud-sig}}
Fedora Cloud Product Requirements Document.
<!-- The code for this page is kept at https://pagure.io/cloud-sig/blob/main/f/PRD.mw and should be edited and merged from there -->
= Document Purpose and Overview =
= Purpose and Demographic =
This is the [http://en.wikipedia.org/wiki/Product_requirements_document Product Requirements Document] for the Fedora Cloud SIG. It:
The ''Cloud Edition Working Group'' targets efforts towards the
* Provides a high-level market of the cloud computing market as it pertains to the Fedora Cloud SIG; this includes overviews of things which may not be within our actual scope/ability to accomplish at the current time.
facilitation and improvement of continuous delivery by ensuring that Fedora project
* Provides deeper understanding of the types of users who could use Fedora for their cloud computing needs. This includes describing their main day-to-day tasks, common problems, and so on. The perspective here is not necessarily limited to system administrators, or developers, but a combination of many types of users and roles.
solutions support cloud native workflow without forcing users to understand all of the
* Ties common issues and needs of potential users and consumers of the Fedora Cloud product to high-level product needs, from a "functional" standpoint
environments in which they require enablement.
* Provides solutions, in the form of "changes" or "features" which will provide the functionality described as needs for the potential users.
= Release/Product Overview =
Fedora Cloud provides a customizable base image and tools for developing scale out applications on public and private clouds, as well as two to four images pre-configured for what we expect to be the most popular scenarios/use cases for using Fedora for cloud computing.
Public and private cloud adoption is taking off, and the requirements for an image OS differ significantly from the requirements for a desktop or server OS. In these environments, much or all of the instance lifecycle — from the creation of the image and addition of software or configuration specific to the instance, to the teardown of the image — will be automated.  Systems are designed to scale out via many identical nodes rather than scale up with carefully-tended individual servers. Individual uptime (mean time between failure) is not as important as the ability to get a new instance running quickly (mean time to recovery).
With that in mind, we're tailoring a release specifically for cloud environments.
== Market Opportunity ==
Public and private cloud adoption is happening rapidly, but the market is not yet mature and is relatively ripe for disruption even though some favorites have emerged as early leaders.
In the next two to three years, we expect to see a great deal of growth in adoption and still see a number of emerging players where no clear favorites have emerged (for instance, Google Compute Engine). Additionally, some platforms (Amazon Web Services) have matured to a point where a large number of companies are relying upon the technology for their full infrastructure. While this is not a widespread practice, AWS is seeing a great deal of adoption and will likely start eating into "traditional" workloads that currently live behind the firewall.
In short, there's an enormous opportunity for Fedora to become an instance-OS of choice if the project moves quickly, develops or adopts the right technologies, and succeeds in educating the market about its existence. A failure on any of those three points means that the Fedora Cloud product will have little chance in taking a significant portion of the new market or taking any of the existing market.  


=== Major Release Themes ===
= Product Objectives =
Cloud computing in general is the transition of computing power from individual hand-tended resources to a ubiquitous utility. Fedora fits in at several levels, from the infrastructure service software we include (like OpenStack and Eucalyptus) to end-user tools.
The ''Fedora Cloud Edition'' allows users to work across multiple virtual environments at
The Fedora Cloud image fits into middle of this, providing a guest OS image to run on Infrastructure as a Service systems, on which platform and application services can be deployed. We targets use cases which fit the "cattle" side of the "[http://www.slideshare.net/randybias/architectures-for-open-and-scalable-clouds pets vs. cattle]" metaphor for computing.
scale by focusing engineering efforts on supplying a base disk images, rpms, and
container-based tooling of various architectures for working in and around public and
private cloud environments.  


=== Secondary Objectives ===
== The Fedora Cloud Base Image ==
Aside from adoption and development of applications on top of the Fedora Cloud images, we have a few secondary goals that will be helped by wider adoption:
There are many requirements for building and deployment into the various options for
* More testing of Fedora images with additional bug reports.
private and public clouds. The primary goal for the Cloud Edition is the Fedora Cloud Base
* Better feedback about how the product should improve. This is separate from "bug reports" in that we hope to engage the audience and receive detailed feedback about use cases, desired features, developing trends in cloud management, etc.
Image. The Cloud Base image provides disk images for all of the environments that can be tested
* Patches and contributions that will help improve the product, and Fedora in general. As we are increasingly successful, more users will take an interest in helping to develop our product.
and confirmed to be functional. It is foundational to additional configurations and
initiatives that require an image-based deployment model.


== Target Market / Audience ==
== Adaptations for Hyperscaler Use Cases ==
Developers and operators creating scale out applications on top of public and private clouds, and organizations and users running those applications. Fedora is particularly interested in organizations that might want to contribute back to Fedora and be involved in helping find/report/fix bugs, and develop new features.
Some use cases may be considered as antithetical to the goals of other edition deployment
== Delivery Mechanisms ==
models, but have a strong use case to be included by default in hyperscalers. The use of
Cloud images must be easy to consume as part of a public or private cloud. Because we target these environments, we won't be worrying about physical media at all. The cloud images also won't be "installed" in the same manner that users are accustomed to with desktop or server images. The cloud image will simply boot in its target environment ready to run, or for further customization and configuration via a boostrapping service.
cloud-native components is continually evolving and highly opinionated, so there is added
=== Where to obtain ===
benefit to including these curated images for use in the exploration of new technologies
Users will be able to obtain the images for public clouds via download ''or'' via the usual marketplace for those images. For instance, we publish AMIs to be run on the [https://aws.amazon.com/ Amazon Web Services] (AWS) directly to the [https://aws.amazon.com/marketplace AWS Marketplace]. Users are able to launch new instances with Fedora without having to obtain the images directly from the Fedora Project and then upload to AWS.
and scientific reproducibility.
Users will be able to download appropriate images for Apache CloudStack, Eucalyptus, OpenStack, OpenNebula, and other IaaS platforms.


=== Delivery Format ===
At the 2020 Power Management and Scheduling in the Linux Kernel summit (OSPM)[1], Andrea
Images will be delivered as AMIs on Amazon EC2, and as downloadable images in qcow2 and raw.xz formats. We may add other public cloud images and other downloadable formats to meet demand or anticipated need.
Righi and Rafael Wysocki discussed the use of hibernation on cloud-based systems and
=== Architectures ===
reviewed work that they have done in support of the program. Jonathan Corbet points out
Initially, we will target 32 and 64-bit x86. As ARM-based cloud systems become available, we will add ARM images. We may consider dropping 32-bit x86 in the future.
that "[t]he value there is to be able to pause a workload to save money. For example,
=== Updates ===
Amazon's ''spot instances'' run at low priority when there are spare resources available;
Fedora Cloud images can be updated using yum as normal. We also intend to produce periodic respun images with security updates, once the infrastructure is in place to support that.
they can be shut down with ten minutes notice at any time."  With support from the Cloud
=== Image Creation Toolkit ===
Edition team, the agents and kernel support can be maintained consistently and
We will also maintain a set of tools that can be used to generate, modify, and configure Fedora instances for use with public and private clouds. Initial efforts will focus on creating official images in Fedora's build system. This effort is in parallel to that and does not block the main release. The Fedora Cloud SIG is also interested in a curated library of image templates.
reliably.
Unofficial templates will be made available through a peer-reviewed portal, so that users can use these templates with confidence to bootstrap their own images.


== Measuring Success ==
There are a number of example use cases to explore in both traditional and non-traditional
Currently, Fedora is not a widely used option for instances on public and private clouds. We know there's some usage, but it's not one of the top three or four OSes on Amazon or (likely) for private clouds.
ways and some of them are included in the next section. This use cases section is intended
Success looks like:
to be updated as time goes on to identify specific initiatives and configurations that the
* Increase in adoption.
Cloud Edition can support.
* Third party support / targeting of Fedora Cloud as a platform.
* Increased contribution and participation in the Fedora Cloud WG and Fedora Project in general.
= User Profiles, Goals, and Primary Use Cases =
== User Profiles ==
There are three cloud user roles which describe the tasks of the people who interact with any cloud-based system:
# Cloud Service Creator: Develops the technical and business aspects of a (simple or high-level) cloud service
# Cloud Service Provider: Provides all types of services to a Service Consumer
# Cloud Service Consumer: Consumes all types of services offered by a Service Provider
We will use a set of personas  to describe our target users and their respective needs. These users are primarily in the cloud service creator and cloud service provider roles.
This document lists the personas in summary form, with detailed  explanations of each one available in a separate document. https://fedoraproject.org/wiki/Cloud_Personas. These roles include aspects of cloud service creators, providers, and consumer.
# AWS enthusiast and Early Adopter
#* Writes and maintains a number of AWS-deployed applications including staging and production.
#* Works for a small start-up, primarily on his own. Automates as much as possible.
#  DevOps Team Member
#* Uses the cloud to do Ruby on Rails development utilizing virtual machines.
#* Works as part of a DevOps team responsible for all aspects of a set of applications.
#* Development, QA, and Production environments need to be identical.
# Web Developer
#* Concentrates on app development, not architecture, deployment, or operations.
#* Unlikely to care deeply about OS internals, cares that they can easily deploy the frameworks they want to use.
# Large-Environment Sysadmin
#* Interested in deploying self-service PaaS to lighten operational load
#* Will work closely with development team on deployment and development.
# Data Scientist
#* Works with large data sets
#* Interested in big data tools and dedicated scripting languages (R, Octave, etc.)
# HPC Scientist
#* Moving from traditional batch and grid technology to the cloud.
#* Interested in hybrid cloud (infrastructure overflow) for some massive computation campaigns.
== Primary Use Cases ==
=== Web Application Deployment in a Public Cloud ===
Modern web applications are deployed as a collection of interconnected services, including parts like web servers, application servers, databases, and caching layers. Fault tolerance is handled at the overall orchestration level rather than by individual instances. Fedora Cloud can be the base of each of these parts, providing recent libraries, server software, and language toolchains. Each system will be managed using the public cloud's own management tools plus a configuration management system like Puppet, Chef, Salt, or Ansible.
Code may be developed against different language stacks; if these are readily available to users, that's a win. In the real world, libraries and other code we don't include may also be required and we should make that as painless as possible.
Fedora Cloud will provide access to the following stacks (not exhaustive):
* Ruby: Rails, Sinatra
* Python 2 & 3: Django, Pyramid, Flask
* PHP 5.x: Symfony2, ZendFramework 2.x
* Node.js
* Java
* Perl 5.x
* Go


=== Web Application Deployment in Private Cloud ===  
== Example Use Cases ==
As above, but in a locally-deployed and managed private cloud system.
=== An Example: Base image for use with Cloud IDE ===
=== Web Application Deployment in Hybrid Cloud ===  
The use of Fedora as a foundation for IDE environments such as the Amazon ''Cloud9'' IDE
As above, but rather than a single cloud provider, the application seamlessly takes advantage of resources in both a private and public cloud.
requires curation and support. When it is not properly prepared, it cannot be included as
=== Hybrid Cloud as Staging Area ===
a part of the product offerings available to users who might wish to use them. A curated
Application development and testing happens in-house on a private cloud, while production goes to a public cloud.
package group is beneficial, but insufficient for Amazon service teams to trust that there
Possible scenarios: Using an open source IaaS stack internally and externally (e.g. OpenStack internally, Rackspace externally; Eucalyptus internally, AWS externally; CloudStack internally, any of the CloudStack providers or AWS externally; or using a library like jClouds to smooth out the bumps.) This may include identical which run in different clouds, or cloud-specific tailored images onto which an application is deployed, or even very different environments under which the same Docker container is run.
will be continued support. The Cloud Edition is consistent with other Fedora Editions, but
=== Big Data / Number-Crunching ===
includes some active modifications to ensure support for next-generation technologies that
Deploy scale-out application for data processing to public, private, or hybrid cloud for running batch jobs. This includes short-lived instances that will be used for processing a large amount of data in a scale-out fashion; perhaps using Amazon Web Services spot instances or similar instances on other providers.
would not be possible with systems intended to be more consistent with downstream
Possible scenario: Using Fedora plus Hadoop and other Hadoop ecosystem pieces to do processing in a public/private cloud setting.
directives.
=== PaaS node images ===
We should be able to provide Fedora Cloud Images for platform-as-a-service (ie: OpenShift Origin) for easy deployment.
Since we have a limited output, Cloud SIG will focus on a limited set of open-source PaaS, and provide support for volunteer-based efforts to port other PaaSes.
Currently supported:
* OpenShift Origin (included in Fedora Packages Collections)
=== Simple Deployment of Code from Development to Production ===
As a developer, I want to ensure that my code is easily deployable from my development environment to a QA environment and finally a production environment, without encountering compatibility issues or surprises from differences in the environment. A solution where an identical customized image or container is used in all three environments is ideal.


= Target IaaS environments =
=== An Example: Base Image optimized for use with cloud native Kubernetes services ===
The Fedora Cloud product can be used as a guest/VM/image under many IaaS services and providers. For projects that are not currently packaged within Fedora/EPEL, we may need to locate a kind person to ensure testing. These include:
Amongst the hyperscaler cloud infrastructures there are those services where a specialized
Open source IaaS systems:
internal community is served. One for which the standard Fedora Editions does not include
* OpenStack
specialized support. Optimizing images for use with downstream platforms, i.e. not OKD or
* Eucalyptus
Red Hat OpenShift, for managing containerized workloads the cloud images can be for
* Apache Cloudstack
example, Kubernetes (k8s): Support for k8s is,in all cloud providers, an opinionated
* OpenNebula
configuration that fits specifically an immutable operations model.
* oVirt
Public clouds:
* Amazon EC2
* Google Compute Engine
* HP Cloud
* Rackspace
* Digital Ocean
* Linode
= Features =
== Feature: Ready to run in Public and Private Clouds ==
The Fedora cloud image is ready to go out of the box in the public and private cloud environments we target. It includes cloud-init, the defacto standard for boot-time configuration for cloud instances, and heat_cfgtools package, which is used for handling guest initialization with OpenStack Heat.
== Feature: Ready Access to Fedora Collection of Packages ==
* Access to the standard Fedora package repositories
* Standard language stacks (like Python, Ruby, NodeJS, PHP)
* Modernized language and application-specific stacks, perhaps in the form of SCLs, for running userspace applications
== Feature: Updated Images ==
Currently, the Fedora Cloud images are only produced twice-yearly as part of the general Fedora release. Because Fedora updates constantly, this means that after several months, it is often the case that a newly-launched image has pending updates which effectively replace the entire content. This is slow and wastes bandwidth, and in many cases means that users simply never update.
Instead, we will produce images with security updates and critical bugfixes rolled in. Our initial target is monthly releases, with special updates produced for critical security flaws.
== Feature: Docker support ==
Docker is an easy to use interface for running application containers on Linux. Fedora is uniquely positioned to provide the best platform for Docker, since this container technology is not a security solution, but can be made reasonably secure when wrapped with SELinux. This includes adding libvirt support to the image, which is more heavyweight than many users of the image for other purposes will want, so we will produce a dedicated image specfically tailored for Docker.
== Feature: Big Data Tools ==
We will produce a specialized image with various tools for Big Data processing preloaded and preconfigured where possible. The exact composition of this image will be determined in collaboration with the Fedora Big Data SIG <ref>[https://lists.fedoraproject.org/pipermail/bigdata/2014-January/000279.html Fedora Big Data SIG vision as discussed on january, 2014 on the big-data list]</ref>.


== Feature: Quick OpenShift Deployment ==
=== An Example: (EDA) Electronic Design Automation ===
We will provide specialized images to deploy quick OpenShift Origin (broker + nodes), with little to no configuration, anyone should be able to deploy a single-node to a simple multi-nodes deployment. The node image should be usable to extend existing OpenShift Origin deployment. The exact composition of this image will be determined in collaboration with OpenShift Origin upstream.
EDA workloads typically target their support towards stable, downstream EL distributions
== Feature: Cloud -> Server ==
and are therefore not capable of moving as quickly as distributions like Fedora. With the
The Fedora Server product targets more traditional server roles, where systems have a more unique identity. The Fedora Cloud image supports this by providing a path to go from our base to a Fedora Server role, in effect taming one of your cloud computing "cattle" and making it into a "pet" traditional server — but running in a cloud environment.
use of cloud-native technologies, it's clear that the design world is at last ready to
== Future Features ==
engage agile processes for the development of system-on-chip (SoC) design. The electronic
The following areas are unlikely to be completely addressed in the upcoming iteration of Fedora Cloud, but we intend to work on them in the future.
design automation (EDA) industry is maneuvering itself into position that makes new
=== Image Generation Tools ===
technologies critical to decreasing time to market. Almost all of the pieces to facilitate
The official Fedora Cloud images will be produced in the Fedora build system (Koji) using kickstart scripts, Anaconda, Oz, and ImageFactory. We also want to provide tools and documentation so users can easily respin their own images. This will likely be using ImageFactory as well. We may provide a web-based service for user-created images.
an SoC methodology are in place in terms of cloud-managed development tools. The industry
=== Image or Image Template Library ===
open access model is an ideal dovetail to ensure that the Fedora Cloud Edition can be a
In addition to the basic tooling for image creation, we want to enable sharing of user-created images and their recipies. This may apply to Docker or other container images in addition to images targetted at full virtualization.
target for early development and adoption of advanced technology.
=== Higher-Level Cloud Tooling ===
Initial efforts focus on individual cloud images. In the future, we want to address higher level concerns as well. These areas include:
# Orchestration
# Performance / Scalability / Failover
# Logging / Auditing
# Monitoring / Notification
# Database requirements


= Requirements =
For EDA, there is an expectation that a significant number of applications will run
== Requirement #1: Reducing Image Footprint ==
generally in the same operating space. Additionally as function specialization occurs, it
Making the Fedora Cloud image fit in a smaller space. Storage is a significant cost element on cloud platforms.
will be beneficial to have an engineering specialization in tailoring these specialized
=== Feature Addressed ===
operating systems to the requirements of software-defined architecture.
Ready to run in Public and Private Clouds.
=== Priority ===
Nice to Have. The current cloud image is reasonably sized when compared to our competitors. However, smaller footprint has several advantages. Fewer packages means fewer updates and a smaller target for security problems. It makes it faster to download and deploy images across networks where the image storage and compute nodes run separately. And, reducing things our users don't really need gives more room to include things that they do.
=== Specific Areas ===
This requirement has several areas where effort will yield meaningful results. Each has its own level of effort, stakeholders, and dependencies.
==== Cloud-Focused Base Kernel ====
The cloud-focused base kernel will be a minimal kernel without many of the drivers traditionally necessary on hardware, like most PCI devices. The goal of having a custom kernel package is ultimately to reduce the size of the images which will reduce storage costs, make the images easier to move across networks between compute and image storage nodes.


Note that the kernel binaries will be shared across all of Fedora; the difference will be that not all modules will be packaged and included by default in the cloud image. This also means that it will be simple to add the other drivers if they are needed for special cases.
== Focus on the familiarity of the users. ==
There are many people who have not advanced to the level of using container-based models
for their bespoke code or operations practices. We don't want to leave them
behind. Furthermore, there are still established practices which can only support
containerization in part. Supporting these transitional and hybrid models is the focus of
the Cloud Working Group.are looking at ways we can build support for next-generation
support models.


Effort is moderate; it includes defining the exact requirements of the smaller kernel, and creation and maintenance of that separation by the Fedora kernel team.
Many users of applications software have decades of experience interacting with a base
operating system standard actions. They may not yet be able to move their practice to an
immutable OS model or use projects that abstract away from decades of \*nix software
development models.


==== Internationalization / Localization ====
== An established practice for investigating the best model for deployment and governance ==
Fedora Cloud base images should only include a single default locale since it is meant to be used as an deployment target to save space. Other locales should be available through repositories. The idea is to rely on langpacks rather than the current RPM kludges which leave out internationalization, because that approach does not allow one to easily add language support later.
In many cases, cloud providers may determine that their opinionated model is the best or
Effort required is fairly high, because the packaging tools do not currently have such support. It will also likely have some impact on all Fedora packagers, depending on the implementation, and may also involve changes to the packaging guidelines.
singularly valid model. This may be the result of focused groups or Product Managers who
==== Included Documentation ====
do not have a familiarity with the possiblities of incorporating open source into their
Documentation has the same issue as internationalization: it is often not needed on each image but takes significant space, and cannot be easily added back if RPM is configured to exclude it initially.
active process due to concerns of any sort. It is a function of the cloud working group as
Effort is similar to that for internationalization, although significant impact could be made by breaking out documentation from key packages included in the base image.
curators of the edition to actively explore these management models and software decisions
==== Refactoring Cloud-Init ====
as a method to either determine the best of class open source interaction or to identify
Cloud Init was initially designed for a different distribution and is only loosely tailored for our needs. As it stands, it pulls in a rather large set of packages not used for other things. It is also written in Python, itself a large subsystem which it would eventually be nice to leave out of the base.
in a way that is acceptable to the greater Fedora community exactly why the current model
Effort is moderate, with some low-hanging fruit which may be addressed easily.
is not handled in an [https://www.theopensourceway.org/ open source way].
== Requirement #2: Improved Docker Integration ==
Docker is a popular tool for launching and managing application containers. This runs on bare metal, in virtualization on developers' machines, or in cloud guests.
Right now, Docker can be added to the cloud base image but isn't included by default. Docker itself is small, but infrastructure to support it includes Libvirt, which is not small. In order to make it easier to get started and to speed up use in production, we will produce a Docker-specific image.
Additionally, we want to create a library of Dockerfiles and possibly our own registry of images.
=== Feature Addressed ===
Docker support.
=== Priority ===
Should. This feature is an important change to support a variety of different teams working on PaaS (like OpenShift) and end-user delivery issues. That being said, since there is very little usage of Docker in those tools right now and the package is available in the repositories already, it won't be terribly detrimental to not have the image complete.
=== Effort required ===
High. We can produce the basic image with minor work, but SELinux is a key feature which will make us stand out from other systems on which Docker can be run. Additionally, producing official Docker container images depends on changes to the build system.
=== Stakeholders / Owners / Major Dependencies ===
The Cloud SIG and our users are the major stakeholders. The SELinux team is instrumental in delivering the functionality we would like to see. Fedora Infrastructure and Release Engineering are involved in updating the build system. And the official images will be produced through Release Engineering.
Major dependencies include:
* Better SElinux MCS separation incorporated in the policies shipped in Fedora.
* Creation of an agreed upon set of Dockerfiles for different tasks.
* ImageFactory support for creating Docker images, and that support integrated into Koji.


== Requirement #3: Automatic Production and Upload of Images ==
Examples of opinionated models are the use of python 2.7 in Google's gcloud sdk or the practice
We will develop Fedora Cloud images using text-based descriptive files and use an automated toolchain that generates images for our targets and upload them where appropriate. Currently, this is done by hand. Automation is necessary to scale to multiple supported images and a growing set of targets.  
of clobbering the systemd-acpid \`sleep.sh\` file in the Amazon Hibernation Agent. Simply
Fedora images must pass automated QA before publication to ensure a minimum level of quality. In order to release updated images with confidence, automated testing is required, because we cannot expect the level of human attention which can be brought to a twice-yearly release.
put, these are problems for the Cloud Edition to resolve and to help work with the
upstream teams responsible to make their applications functional in the context of the
=== Feature Addressed ===
Fedora Distribution and downstream Enterprise Linux.
Updated Images.
=== Priority ===
Must. We simply need this to be useful to our users, and therefore to be competitive.
=== Effort required ===
High. Release Engineering does not currently have the tooling in place to support this.
=== Stakeholders / Owners / Major Dependencies ===
The Fedora Cloud Working Group will work with Release Engineering in implementing and supporting the needed functionality.
== Requirement #4: Needs for Big Data Image  ==
Since large and complex data sets that are difficult to process using RDBMS and traditional data processing tools are now common, and the open source big data ecosystem is now quite mature, we ought to provide a ready-to-go image for these usages.
Hadoop and its ecosystem is clearly a key player and the Big Data SIG is working to bring it on Fedora.
Real-time analysis is also growing, and components like Spark and Storm are also something we'd like to include.
=== Feature Addressed ===
Big Data Tools.
=== Priority ===
Should. Very useful for our users, but does not block anything else.


=== Effort required ===
As a part of the working group, this team builds, monitors and improves the upstream
Medium. Requires packaging the missing bits of the Hadoop ecosystem, most of it being java packages. The effort is being led by the Big Data SIG, as a stakeholder, and the Cloud SIG will support them to improve this.
experiences complicated by closed or non-standard models to ensure that there is a path
=== Stakeholders / Owners /  Major Dependencies ===
forward to remove the barriers that block open source success for the cloud communities to
The Fedora Cloud SIG will work with the Big Data SIG to package and deliver a rock-solid Big Data Image.
which we are aligned.


== Requirement #5: Software Stacks for Various Languages ==
= Architectural Considerations =
People developing and deploying code on top of Fedora need access to the relevant language stacks. Fedora already provides the latest versions of many of these, but often real-world version dependencies do not match the Fedora cycle. We need some way of offering multiple versions of languages and software stacks. Additionally, it is ideal if support for a given version lasts longer than the 13-month Fedora lifecycle, so that moving from one release of the base to the next can be less painful.
== Boot Support ==
The Cloud SIG plan to support the major languages:
* Legacy Bios or ''Classic Boot''
* Ruby
* UEFI
* Python 2 & 3
* PHP 5.x
* Java
* Node.js
* Perl 5.x
* Go
We will provide infrastructure and support to volunteers who want to complete the supported stacks or bring new ones.
=== Feature Addressed ===
Ready Access to Fedora Collection of Packages.
=== Priority ===
Must. This addresses a key need of userbase.
=== Effort required ===
High. Access to existing Fedora packages is simple (we simply won't break what currently works), but Fedora does not currently have a good infrastructure for multiple versions of stacks or for decoupled lifecycles.
=== Stakeholders / Owners / Major Dependencies ===
Software in this area will be the responsibility of the Environments and Stacks working group.


== Requirement #6: Integration with Fedora Server Roles ==
  The Boot support is dependent upon a number of other considedations. Neal Gompa's collaborative work with the Red Hat team has made our images ready for both without modification. Aarch64 is UEFI only.  
We simply want to make sure that the tools to convert a Fedora Cloud base image to a Fedora Server role are available and function.
=== Feature Addressed ===
Cloud -> Server
=== Priority ===
Should.  This helps tie the Fedora products together and makes the blurry lines  between the Cloud and Server product more understandable.
=== Effort required ===
Medium. Effort is primarily in coordination and testing.
=== Stakeholders / Owners / Major Dependencies ===
This effort will be done in collaboration with the Fedora Server Working Group.
== Requirement #7:  Tools for User Image Creation and Image / Template Library ==
We do not have a standardized process for end-user image creation, although we have several tools which can be used. We plan to develop this process and document it. This may also include an image library or a library of image creation templates, or both.
=== Features Addressed ===
Image Generation Tools,  Image or Image Template Library
=== Priority ===
Nice to have. This is not required for our initial launch.
=== Effort required ===
High.
=== Stakeholders / Owners / Major Dependencies ===
Image creation tools (Oz/ImageFactory, etc.), Fedora Websites, Fedora Documentation, Cloud SIG users in general.


== Requirement #8: Updated Web Site for Obtaining Cloud Images ==
== Processor Architecture Support ==
As we provides images for various use cases (ie: Big Data, PaaS hosting, scale-out apps, etc.) and various target platforms, we need an updated website.
* x86_64
Fedora provides its build toolchain and encourage the community to use it to build new products. The updated website should also reflect that by allowing folks to share and review the work done by their peers.
* aarch64
=== Feature(s) Addressed ===
Easy access to Fedora Cloud products and encouraging sharing in the community.
=== Priority ===
Must. A simple access to the cloud products is also a key argument to their success.
Should.  Providing a sharing platform to the community (as in not supported by  the Cloud SIG) and peer-review based system will allow to service better  our users and foster innovation.
=== Effort required ===
High. We need to define precisely what we want to show, and maintain the website.
=== Stakeholders / Owners / Major Dependencies ===
The Cloud SIG will work closely with Fedora Websites/Infrastructure and possibly Fedora Engineering if we need to develop specific apps and/or integrate with the existing Fedora infrastructure (like fedmsg).


== Documentation ==
= Participants =
We aim to provide a comprehensive documentation covering users to developers.
Currently the following people are involved in the Cloud Working group.
Should. Provide narrative documentation to make it easier to use cloud images and use them as foundations to new products.
NTH. On a volunteer basis, we will maintain a knowledge base.
=== Fedora Project Documentation ===
The Cloud SIG will cooperate with the Documentation Project to provide documentation to users, and developers of Fedora Cloud Products.
=== Open Source Projects documentation ===
Ensuring that Fedora is well-represented, up-to-date in other open source project documentation...
= About this Document =
This PRD (Product Requirements Document) is an evolving document, created by the Fedora Cloud SIG Working Group as part of the process for designing the Fedora Cloud product. The framework for the PRD itself is currently in a draft state.
This document will evolve over time as the purpose of the SIG continues to be determined as the working group process gets under way and initial products start to get produced.
== Authors ==
Contributors to this document include:
* [[User:rbergero|Robyn Bergeron]]
* [[User:mattdm|Matthew Miller]]
* [[User:skottler|Sam Kottler]]
* [[User:Hguemar| Haïkel Guémar]]
* [[User:jzb| Joe Brockmeier]]
* [[User:frankieonuonga|Frankie Onuonga]]
== Reviewers & Contributors ==
The following people have contributed to the development of this document, through feedback on IRC, mailing lists, and other points of contact.  
* [[User:mattdm|Matthew Miller]]
* [[User:skottler|Sam Kottler]]
* [[User:Hguemar| Haïkel Guémar]]
* [[User:jzb|Joe Brockmeier]]
* [[User:frankieonuonga|Frankie Onuonga]]
==  Credits ==
Some of the Cloud User Roles are based on “Description and Application of Core Cloud User Roles” ACM CHIMIT 2011, December 4 2011.
Some aspects of Fedora Cloud personas are based on [https://docs.google.com/document/d/16rkiXWxxgzGT47_Wc6hzIPzO2-s2JWAPEKD0gP2mt7E OpenStack personas] (licensed under CC-By 3.0)
==  Community Information ==
The Fedora Cloud SIG is one of many teams within the Fedora Project.
The Cloud SIG mailing list is located [https://admin.fedoraproject.org/mailman/listinfo/cloud here.]
Minutes and logs from IRC meetings related to the development of this document should be listed here as the document evolves.
== Approval History ==
Over time, it is expected that this document will undergo various rounds of review, approval, and editing; in the future, it may be rewritten for different releases of Fedora.
While one can review the history of a wiki document (by clicking the "history" tab), it is useful to provide explicit indicators of any major format changes, approvals, or indications of it being in a “final” state, in a list that can allow someone to quickly see that all of the prescribed layers of approval have occured.
* January  8, 2014: Collaborative hackfest day.
* October 28, 2013: [http://fedoraproject.org/w/index.php?title=Cloud_PRD&oldid=358260 Initial Draft of template.]
* FutureDate: Approval by SomeGroup; link to any pertinent mail announcement and/or meeting minutes
== Document Conventions==
=== Definitions and Acronyms ===
* AWS: Amazon Web Services
* Amazon EC2: Amazon Elastic Compute Cloud, a popular public IaaS
* IaaS: Infrastructure as a Service
* PaaS: Platform as a Service
* SaaS: Software as a Service
* PRD: Product Requirements Document
* [[EPEL]]: Extra Packages for Enterprise Linux
* CI: Continuous Integration
* CD: Continuous Delivery or Continuous Deployment
* [[SoftwareCollections | SCL]]: Software Collections
* [[Category:SIGs|Special Interests Groups]]: teams in charge of some aspects of Fedora Project
* NTH: Nice-to-have
* BZ: [[Bugzilla]]
* GUI: Graphical User Interface
* CLI: Command Line Interface
* API: Application Programming Interface
* RDBMS: Relational DataBase Management System


* [https://fedoraproject.org/wiki/User:Davdunc David Duncan]


<references />
* [https://fedoraproject.org/wiki/User:Dustymabe Dusty Mabe]
[[Category:Product_Requirements_Document]]
 
* [https://fedoraproject.org/wiki/User:Mhayden Major Hayden]
 
* [https://fedoraproject.org/wiki/User:Ngompa Neal Gompa]
 
* [https://fedoraproject.org/wiki/User:Dcavalca Davida Cavalca]
 
* [https://fedoraproject.org/wiki/User:Salimma Michel Salim]
 
* [https://fedoraproject.org/wiki/User:Chrismurphy Chris Murphy]
 
* [https://blog.centos.org/2022/03/new-centos-director-amy-marrich/ Amy Marrich]
 
* [https://fedoramagazine.org/joe-doss-how-do-you-fedora/ Joe Doss]
 
Many others have participated in the past and the projects a whole was
founded by the current (2022) [https://docs.fedoraproject.org/en-US/council/fpl/ FPL], [https://fedoraproject.org/wiki/User:Mattdm Matthew Miller]. Matthew has been integral in
working on the scope and practices followed by the Cloud Working Group for many years.
 
 
= Project Status =
The Working group is '''ACTIVE'''
 
Currently the Cloud Working Group is focused on returning to status as
an [https://fedoraproject.org/wiki/Changes/RestoreCloudEdition Edition].  Edition status lapsed as a result of a number of
changes. The first was a move to focus efforts related to cloud images
to include [https://projectatomic.io/ Project Atomic] as well as the standard cloud edition and
then a complication occurred when Fedora Atomic was replaced by Fedora
CoreOS. The directives for these working groups is distinctly
different and supports different goals related to downstream adoption
and support.
 
== Meeting Times ==
Bi-Weekly, every other Thursday. Currently we are supposed to be meeting at 15:00 UTC, but we have been meeting accidentally at 8:00 AM EDT daylight savings time. We are considering meeting on the 1st and 3rd of the month with a Video meeting on the 5th week of months that have a 5th week. Changing the meeting time would allow us to synchronize with the CentOS Cloud SIG for consistency.
 
----
=== Footnotes ===
 
[1] https://lwn.net/Articles/821158/n

Latest revision as of 20:45, 1 November 2022

Purpose and Demographic

The Cloud Edition Working Group targets efforts towards the facilitation and improvement of continuous delivery by ensuring that Fedora project solutions support cloud native workflow without forcing users to understand all of the environments in which they require enablement.

Product Objectives

The Fedora Cloud Edition allows users to work across multiple virtual environments at scale by focusing engineering efforts on supplying a base disk images, rpms, and container-based tooling of various architectures for working in and around public and private cloud environments.

The Fedora Cloud Base Image

There are many requirements for building and deployment into the various options for private and public clouds. The primary goal for the Cloud Edition is the Fedora Cloud Base Image. The Cloud Base image provides disk images for all of the environments that can be tested and confirmed to be functional. It is foundational to additional configurations and initiatives that require an image-based deployment model.

Adaptations for Hyperscaler Use Cases

Some use cases may be considered as antithetical to the goals of other edition deployment models, but have a strong use case to be included by default in hyperscalers. The use of cloud-native components is continually evolving and highly opinionated, so there is added benefit to including these curated images for use in the exploration of new technologies and scientific reproducibility.

At the 2020 Power Management and Scheduling in the Linux Kernel summit (OSPM)[1], Andrea Righi and Rafael Wysocki discussed the use of hibernation on cloud-based systems and reviewed work that they have done in support of the program. Jonathan Corbet points out that "[t]he value there is to be able to pause a workload to save money. For example, Amazon's spot instances run at low priority when there are spare resources available; they can be shut down with ten minutes notice at any time." With support from the Cloud Edition team, the agents and kernel support can be maintained consistently and reliably.

There are a number of example use cases to explore in both traditional and non-traditional ways and some of them are included in the next section. This use cases section is intended to be updated as time goes on to identify specific initiatives and configurations that the Cloud Edition can support.

Example Use Cases

An Example: Base image for use with Cloud IDE

The use of Fedora as a foundation for IDE environments such as the Amazon Cloud9 IDE requires curation and support. When it is not properly prepared, it cannot be included as a part of the product offerings available to users who might wish to use them. A curated package group is beneficial, but insufficient for Amazon service teams to trust that there will be continued support. The Cloud Edition is consistent with other Fedora Editions, but includes some active modifications to ensure support for next-generation technologies that would not be possible with systems intended to be more consistent with downstream directives.

An Example: Base Image optimized for use with cloud native Kubernetes services

Amongst the hyperscaler cloud infrastructures there are those services where a specialized internal community is served. One for which the standard Fedora Editions does not include specialized support. Optimizing images for use with downstream platforms, i.e. not OKD or Red Hat OpenShift, for managing containerized workloads the cloud images can be for example, Kubernetes (k8s): Support for k8s is,in all cloud providers, an opinionated configuration that fits specifically an immutable operations model.

An Example: (EDA) Electronic Design Automation

EDA workloads typically target their support towards stable, downstream EL distributions and are therefore not capable of moving as quickly as distributions like Fedora. With the use of cloud-native technologies, it's clear that the design world is at last ready to engage agile processes for the development of system-on-chip (SoC) design. The electronic design automation (EDA) industry is maneuvering itself into position that makes new technologies critical to decreasing time to market. Almost all of the pieces to facilitate an SoC methodology are in place in terms of cloud-managed development tools. The industry open access model is an ideal dovetail to ensure that the Fedora Cloud Edition can be a target for early development and adoption of advanced technology.

For EDA, there is an expectation that a significant number of applications will run generally in the same operating space. Additionally as function specialization occurs, it will be beneficial to have an engineering specialization in tailoring these specialized operating systems to the requirements of software-defined architecture.

Focus on the familiarity of the users.

There are many people who have not advanced to the level of using container-based models for their bespoke code or operations practices. We don't want to leave them behind. Furthermore, there are still established practices which can only support containerization in part. Supporting these transitional and hybrid models is the focus of the Cloud Working Group.are looking at ways we can build support for next-generation support models.

Many users of applications software have decades of experience interacting with a base operating system standard actions. They may not yet be able to move their practice to an immutable OS model or use projects that abstract away from decades of \*nix software development models.

An established practice for investigating the best model for deployment and governance

In many cases, cloud providers may determine that their opinionated model is the best or singularly valid model. This may be the result of focused groups or Product Managers who do not have a familiarity with the possiblities of incorporating open source into their active process due to concerns of any sort. It is a function of the cloud working group as curators of the edition to actively explore these management models and software decisions as a method to either determine the best of class open source interaction or to identify in a way that is acceptable to the greater Fedora community exactly why the current model is not handled in an open source way.

Examples of opinionated models are the use of python 2.7 in Google's gcloud sdk or the practice of clobbering the systemd-acpid `sleep.sh` file in the Amazon Hibernation Agent. Simply put, these are problems for the Cloud Edition to resolve and to help work with the upstream teams responsible to make their applications functional in the context of the Fedora Distribution and downstream Enterprise Linux.

As a part of the working group, this team builds, monitors and improves the upstream experiences complicated by closed or non-standard models to ensure that there is a path forward to remove the barriers that block open source success for the cloud communities to which we are aligned.

Architectural Considerations

Boot Support

  • Legacy Bios or Classic Boot
  • UEFI
 The Boot support is dependent upon a number of other considedations. Neal Gompa's collaborative work with the Red Hat team has made our images ready for both without modification. Aarch64 is UEFI only.

Processor Architecture Support

  • x86_64
  • aarch64

Participants

Currently the following people are involved in the Cloud Working group.

Many others have participated in the past and the projects a whole was founded by the current (2022) FPL, Matthew Miller. Matthew has been integral in working on the scope and practices followed by the Cloud Working Group for many years.


Project Status

The Working group is ACTIVE

Currently the Cloud Working Group is focused on returning to status as an Edition. Edition status lapsed as a result of a number of changes. The first was a move to focus efforts related to cloud images to include Project Atomic as well as the standard cloud edition and then a complication occurred when Fedora Atomic was replaced by Fedora CoreOS. The directives for these working groups is distinctly different and supports different goals related to downstream adoption and support.

Meeting Times

Bi-Weekly, every other Thursday. Currently we are supposed to be meeting at 15:00 UTC, but we have been meeting accidentally at 8:00 AM EDT daylight savings time. We are considering meeting on the 1st and 3rd of the month with a Video meeting on the 5th week of months that have a 5th week. Changing the meeting time would allow us to synchronize with the CentOS Cloud SIG for consistency.

Footnotes

[1] https://lwn.net/Articles/821158/n

Retrieved from "https://fedoraproject.org/w/index.php?title=Cloud/Cloud_PRD&oldid=659960"