User:Parasense/bodhi2-workflow

Bodhi Backend Tasks

# get the errors
sudo journalctl --since=yesterday -o short -u fedmsg-hub > ~/error.out
awk '/E[Rr][Rr]/' ~/error.out

# or with color
egrep 'ERROR|Errno' ~/error.out

# reset the fedmsg-hub service
sudo systemctl restart fedmsg-hub

# Look for TMPFS relics from rpm-ostree
findmnt -t tmpfs -o TARGET | grep rpm-ostree

# Dismount the relic TMPFS
sudo umount -R -t tmpfs /var/lib/mock
sudo umount /var/lib/mock/*/root/var/tmp/rpm-ostree.??????

# start the bodhi push for signing (responding 'no' when prompted)
cd /var/cache/sigul
yes 'no' | sudo -u masher -S bodhi-push --releases '23 22 21 5 6 7' --username parasense

# sign the push
for i in 23 22 21 ; do NSS_HASH_ALG_SUPPORT=+MD5 ~/releng/scripts/sigulsign_unsigned.py fedora-$i -v --write-all --sigul-batch-size=25 $(cat /var/cache/sigul/{Stable,Testing}-F${i})    ; done
for i in  7  6  5 ; do NSS_HASH_ALG_SUPPORT=+MD5 ~/releng/scripts/sigulsign_unsigned.py epel-$i   -v --write-all --sigul-batch-size=25 $(cat /var/cache/sigul/{Stable,Testing}-*EL-${i}) ; done

# Inspect the repo cache areas (optional)
# Areas to verify empty ?
/mnt/fedora_koji/koji/mash/updates/dist-5E-epel{,-testing}.repocache/repodata/
/mnt/fedora_koji/koji/mash/updates/dist-6E-epel{,-testing}.repocache/repodata/
/mnt/fedora_koji/koji/mash/updates/epel7{,-testing}.repocache/repodata
/mnt/fedora_koji/koji/mash/updates/f21-updates{,-testing}.repocache/repodata
/mnt/fedora_koji/koji/mash/updates/f22-updates{,-testing}.repocache/repodata
/mnt/fedora_koji/koji/mash/updates/f23-updates{,-testing}.repocache/repodata

# Check for existing masher locks of currently running push, or failed previous push
ls -l /mnt/koji/mash/updates/MASHING-*

# Check for running bodhi2 push (via masher)
pgrep -af /usr/bin/mash

# Also check for rsync
pgrep -af rsync

# resume the push interactivly
sudo -u masher bodhi-push --resume --username parasense

# resume the push responding yes to everything
yes| sudo -u masher -S bodhi-push --resume --username parasense

# Follow the output of fedmsg-hub
sudo journalctl -o short -u fedmsg-hub -l -f

Sign Bridge Tasks

# monitor the signing on bridge for potential stalls from bodhi-backend
ssh -v -o'ControlPath=none' sign-bridge01 'tail -f /var/log/sigul_bridge.log'

# Verify the bridge is running or not
pgrep -af bridge.py

# Restart the bridge as necessary 
sudo pkill -f -9 bridge.py
sudo NSS_HASH_ALG_SUPPORT=+MD5 sigul_bridge -d -v -v

# Review the bridge.py output
tail -f /var/log/sigul_bridge.log

Stable push requests

Sometimes a special request to have something pushed to stable, usual security thing, or some othe urgent need.

Here we had two lorax builds in the testing queue, and QA engineer requested they go to stable ASAP.

bodhi-push --releases '21 22' --request=stable  --builds lorax-21.34-1.fc21 lorax-22.13-1.fc22 --username parasense