"Authenticates users by Fedora certificate, verifies they are in a 'signing server users' group" - It would be better to use a hardware certificate for this step. Should definitely talk to Bob Relyea <rrelyea@redhat.com>. He helped write the "coolkey" pkcs11 smart card driver for Axalto Smartcards (and helped write the hardware driven firmware that the cards are initialized with, I believe). If you give these cards are given out to the signers then if a signers machine gets owned, packages will only be able to be signed when the card is plugged in (you could take this a step further by having the signer use a read-only custom live image "client appliance").
The important bit is the signer never has access to the certificate, and so the certificate won't be able to get duplicated into malicious hands.