Changes/ProductDefinitionCenter

These are notes from a "composedb" brainstorming session on August 5th, 16:00UTC.

The initial idea for "composedb" was to have something that knows what goes into every compose and what comes out of it: the atomic repos, the live cds, etc.. what's in them. what's in cloud, server, workstation, etc. We need such a thing so that we have a place where we can go and ask what changed between this compose and this compose, so we can easily visualize what's different between primary arch composes and secondary arch composes, etc.

Furthermore, it would be a more robust solution than the releng dash to be able to show (for instance) when was the last nightly compose that worked and to give you a way to visualize when the last updates push was done.

Action We're going to look at PDC (Production Definition Center) for this. It's a Django app that does almost all of that described above.

It may not currently be able to support the notion of "rings" (a la Fedora.NEXT). We need a way to say what's in the different rings (so they can have different policies and processes) And there's lots of things that can be built using this information that we can't do today.

---

Beyond having a system that knows what inputs go into which releng artifacts, it would be great to then develop tooling around that data source. For instance:

• it would be cool if when we're doing the rawhide compose we can look at see that nothing has changed in XFCE so we don't rebuild that livecd, but we do rebuild other artifacts where things actually changed.
• furthermore, with that kind of knowledge we can rebuild artifacts as their inputs change (fedmsg) instead of doing things on a nightly or semi-annual basis like we do now.
• it would be cool to produce reports on the different editions and their artifacts over time. i.e., show how the size of the workstation image is growing (so we can fix it) or show how the size of the cloud image is shrinking (so we can celebrate).
• it would be cool to automatically impose gating via taskotron for some artifacts, depending on what "rings" (Fedora.NEXT) the inputs are in and what policies we have associated with those rings.
• leverage taskotron QA checks to create side-tags where we automatically rebuild stuff in the event of soname bumps. We could then also auto-gate artifacts and keep them from reaching the next step in the process if (for instance) things fail depcheck. Say, stuff in ring 0 and ring 1 require tests X, Y, and Z, but ring 2 requires less. we could make sure that "rawhide is never broken".
• it could be auspicious to build artifacts immediately (as their inputs change) but to gate publication to the mirrors on some sort of human sign-off from releng.

If PDC is the system that knows what goes into what, then this second section is about a system (or group of systems) that knows how to build those things -- and does it. We're calling that system Outhouse. Think of it as a rewrite of the collection of shell scripts in the releng repo into a continuously-running daemon.

However this ends up shaping up, a note about builders:

• We can't use jenkins for building - its unsupported.
• We can't use taskotron for building - nodes in the QA network are of an insufficient security grade.
• We could roll our own thing.
• Or we could kick off everything as koji tasks. This is preferable and we can lay the groundwork for it now by porting our existing compose processes to koji tasks.