Infrastructure 2020/Websites

Document Details

• Status:WORKING DRAFT:
• Point of contact: relrod, ryanlerch
• Implementation Schedule: TBD
• Release Date: TBD

Goals

We want to rework how the Fedora websites are used (by end-users), developed and maintained (by current and potential contributors), and (perhaps) deployed (by the infrastructure team).

In particular, this project consists of (at least) the following elements:

• Reexamining our tooling and switching to a newer static site generator
• Working closely with the design team to establish a minimal set of templates to be used across all of our landing sites
• Working closely with the design team to migrate content into those new templates, untangling hacks from previous renditions of the websites as we go

The section below explains some of the reasoning. However, here we provide a short overview of some benefits to the Project, broken into user and developer targets:

For users:

• Re-alignment of what we wish to highlight on the sites ultimately makes it faster and easier for users to find what we want them to find, and what they want to find. (Hopefully those are one and the same!)
• During the design we can put new emphasis on things such as mobile design (see section below on this)
• A unified look across our sites provides a feeling of consistency around the project and its family of subprojects
• A unified look also makes it easy for users to navigate all of our sites intuitively
• Optimizing how things like assets are delievered (e.g. asset bundling) creates a faster loading experience for users

For developers:

• Having the templates be defined in one place and simply referenced in the various landing sites, means no more copy-pasting of code and losing consistency as one site gets updated assets but not the others
• Using more modern tooling likely makes it easier for contributors to on-board because they are more familiar with it
• Relatedly, having templates not use Genshi almost certainly means easier fly-by contributions for one-off fixes.

Background and Strategic Fit

Fedora's various landing websites represent our primary online presence. They are where new and old users alike go to download the latest Fedora versions, where potential contributors go to see what the Project is all about, and where attendees of events like Flock go to see the latest up-to-date information.

On the frontend side of this, there are things we can do to make it clear to users which parts and subprojects of Fedora we want to highlight. As the goals, users, and needs of the project has changed over the last several years, the websites have not been updated to account for the changes. For example, projects like IOT and Silverblue are barely highlighted anywhere on the site, yet are likely to be interesting and important to our userbase and those interested in the project.

We can also make better use of modern mobile-first design techniques. As things stand, we make poor use of Bootstrap's responsive system by repeating whole sections of code and content for various devices.

On the backend/generation side of this, over time several problems have cropped up with the current system. An incomplete list follows:

• Various "hacks" have been added into our codebase to handle things that were meant to be done for "one release" but never ended up getting dropped
  • Among these are release-specific URLs for certain images (think "oh crap the image didn't build, we will build it separately and upload it somewhere instead for release day and just link to that"). We should either instill a "no exceptions" policy for URLs or account for this kind of thing from the start.
• Website templates have become inconsistent due to not having a common included theme (the theme was instead copy-pasted between various sites, meaning when it gets updated, it must be updated across all sites individually, which just does not happen)
• Contributing even seemingly-small patches to the websites project requires an intimate understanding of how a large number of components and hacks fit together
• Learning Genshi is hard to do, compared to modern alternatives

The principle of this proposal is to explore modern tooling to maximize our various websites' utility and ease of use to users while making it easier for new and existing contributors to make changes to the sites.

People involved

• Backend and infrastructure: Rick Elrod
• Backend POC and design implementation: Ryan Lerch
• Design mockups: Mo?
• Translations: We will eventually need a contact here so they know what to expect with the new site.
• Releng: We will eventually need a contact here so we coordinate on things like download image URLs going foward (they seem to change sometimes)

We will likely want at least one contact with any groups we intend to highlight on the site:

• IOT: We will eventually need a contact here for content and image links
• Cloud: We will eventually need a contact here for content and image links (think: aws images)
• CoreOS: Do we need a contact here?
• Server WG: ?
• Workstation WG: ?

Assumptions and Questions

Assumptions

We aren't tied to the old way of doing things

The goal is to explore new technology and pick the right tool for the job. We should not be tied to things because it's "how they are."

Questions

What infrastructure changes will be needed?

• Packages installed (created?) for new tools we use
• Explore OpenShift or other fancy things vs how the deploy currently works
• Push-based deploys rather than polling for changes to the repo every hour?
  • Combining the two above, maybe consider OpenShift (for push-based generation), then simply using this as an origin for Amazon CloudFront?

What are priorities in looking for tooling?

• easy-to-use template language
• excellent i18n support
• Nice-to-have: Asset bundling for quicker page loads

Content Scope

• What sites will be owned by fedora-websites going forward?
• What sites currently in the repository are unused, or should be retired?

Additional Links

• Current websites repo: https://pagure.io/fedora-websites/
• Exploratory repo (relrod): https://github.com/relrod/fedora-websites-ng/
• Exploratory repo (ryanlerch): https://github.com/ryanlerch/fedora-websites-ng/
• Jinja: http://jinja.pocoo.org/docs/2.10/

Initial Stories

A minimum user story list should be prepared here to facilitate Task Breakdown. The user story describes the type of user, what they want and why. A user story helps to create a simplified description of a requirement. Format should be:

As a < type of user >, I want < some goal > so that < some reason >.

Personas:

• New User: A user new to Fedora (perhaps Linux), not familiar with our ecosystem whatsoever
• Existing User: An end user of Fedora, perhaps looking for a copy of the latest version or seeing what kinds of releases we put out nowadays since their last upgrade
• Existing Websites Contributor: A member of the websites group
• New Websites Contributor: Someone who would like to contribute to the websites project
• Fly-by Websites Contributor: Someone who does not usually contribute to the websites project but wants to make a quick change because they notice an error

Story List

• As a New User, I want to quickly learn what this "Fedora" or "Linux" thing is about before I lose interest
  • I want to easily download a copy of it for my architecture
  • I want to easily be able to navigate to documentation and see the latest Common FXX Bugs wiki page to see if my problem is listed.
  • I want quick access to the community (ask.fp.o, IRC, etc.) when something goes wrong.
  • Perhaps while waiting for the download, I want to learn more about the project because I am excited.
  • I want the site to load quickly because I am impatient.

• As an Existing User, I want to quickly download a the latest version of my favorite distribution for my architecture
  • I want to see all the latest subprojects (think: silverblue, IOT) because maybe I can use one for my latest project!
  • I want to learn about where I can contribute because I <3 the Project.
  • I want the site to load quickly because I am impatient.

• As an Existing Websites Contributor, I want to be able to take a week break from the team and come back and not have to re-learn the entire codebase because of all of the old remnants of bits of code that have conglomerated into a huge pile of rubbery spaghetti code that makes it impossible to understand what is happening and why that is, so that I can change a word on one of the websites without wanting to rip my eyes out.
  • I want to look at a component of the website and easily understand it and understand what happens when I change it.

• As a New Websites Contributor, I want to easily get a feel for how the websites get compiled into the final result.
  • I want to be able to make a meaningful change without too much hand-holding.
  • I want to use my existing knowledge from current technologies and either apply it directly or draw from it easily.

• As a Fly-by Websites Contributor, I want to be able to fix a typo I see on getfedora.org
  • I want to easily find the file that became the page I am looking at, and submit a fix for it

Content Scope

The current Fedora Websites repo contains 17 different sites. These fall into 3 broad categories:

Sites to keep

8 Sites to keep are current and updated and the target for changes to the generation process. The proposed list is:

• https://getfedora.org
• https://alt.fedoraproject.org
• https://arm.fedoraproject.org
• https://flocktofedora.org
• https://iot.fedoraproject.org
• https://labs.fedoraproject.org
• https://spins.fedoraproject.org
• https://start.fedoraproject.org

Already retired sites

These are 7 sites that have been retired, either by being obsoleted by another tool, or just retired. delete these from the repo

• https://budget.fedoraproject.org - this is now at https://pagure.io/fedora-budget
• https://fedorahosted.org - this is now just a redirect to Infrastructure/Fedorahosted-retirement
• https://fedorapeople.org - the source in this repo does not match what is on the site. Assumed it is elsewhere, probably as an ansible config
• https://fedoraproject.org - now just a redirect to getfedora.org
• https://mirrors.fedoraproject.org - replaced by mirrormanager
• https://serverbeach1.fedoraproject.org - not in use, not sure what this was/is
• https://fudcon.fedoraproject.org - no longer in use, and just redirects to the flock website.

Candidates for Retirement

2 sites that are candidates for retirement going forward.

• https://boot.fedoraproject.org - pretty much a FAQ for FBO. Not sure if FBO is still active. If inactive, retire without replacement. If active, retire, moving to docs.
• https://fedoracommunity.org - the content for this site would be better suited for community docs, or the wiki (see https://pagure.io/fedora-websites/issue/936)

Task breakdown

This is a shorthand list of items we are needing to cover for Websites.next in Fedora Websites for the Fiscal Year 2020.

Frontend

• Design team already has mockups for getfedora.org
• Need mockup for the smaller sites and subpages (think flocktofedora.org, iot.fedoraproject.org, silverblue.fedoraproject.org, cloud.fedoraproject.org, etc.)

General Acceptance Criteria:

• (TODO)

Backend/generation

• We need to explore tools and see which ones fill our needs the best
• There are some Python scripts (namely for getfedora.org) that need to port over to the new system
• Consider doing some kind of usability testing.
  • At a minimum, we should aim for making the website load quickly, and explore things like asset bundling

General Acceptance Criteria:

• (TODO)

Infrastructure

• Make any infrastructure changes necessary based on the tooling we pick
• Consider revamping the deploy process (consider OpenShift, Amazon CloudFront, etc.)
  • Push-based deploys to eliminate polling

General Acceptance Criteria:

• (TODO)

Future Plans

Phase 0

Phase 0 is the exploratory phase, which has already started (see the relevant links above). In this phase we compare and contrast several tools and see which meets our needs the best. Design work continues.

Phase 1

Phase 1 is picking a contender from Phase 0 and running with it. In this phase, we start doing the actual implementation and moving content over (rewriting as needed/wanted), and begin finalizing the design.

Phase 1 consists of getting most of the work on these sites done:

• getfedora.org
• flocktofedora.org

(These are subject to change, but the reason I picked these is because getfedora.org needs to be done for F30/F31, and flocktofedora.org gives us a smaller site alongside to test that whatever system we decide upon works with multiple sites, along with providing a testing ground for the small-site template. There is a chance that this is taking on too much, though.

Phase 2

Phase 2 consists of getting translations for the above sites completed, while any infrastructure changes decided upon are made to get them ready for deploy to staging.

Phase 3

Phase 3 is the deployment of the above sites to staging, and asking users to test the heck out of them.

Phase 4

Phase 4 is the deployment of the above sites to production and should correspond with (??? F31?).

Phase 5

Phase 5 is using the templates to redo the rest of the static sites, then deploying them similarly to above. Once the workflow is ironed out for the above sites, this phase should be relatively smooth-sailing.