From Fedora Project Wiki

< User:Hamzy
Revision as of 16:36, 5 November 2017 by Hamzy (talk | contribs)

Creating an overcloud image builder instance

Start a QEMU instance

Setup the anaconda kickstart file. 

[hamzy@pkvmci853 ~]$ cat << '__EOF__' > libvirt/anaconda-ks-overcloud.cfg
#version=DEVEL
# System authorization information
auth --enableshadow --passalgo=sha512
# Use network installation
url --url="http://mirror.centos.org/altarch/7/os/ppc64le/"
# Use text mode install
text
# Run the Setup Agent on first boot
firstboot --enable
ignoredisk --only-use=vda
# Keyboard layouts
keyboard --vckeymap=us --xlayouts=''
# System language
lang en_US.UTF-8

# Network information
network  --bootproto=dhcp --device=eth0 --ipv6=auto --activate
network  --hostname=overcloud.virbr0

# Root password
rootpw --iscrypted $6$>?@N4]-Ujl.o~$lUoOP3vNc3Q.D2m4m0Z.wQBC.nMTEvclR.T./xx7slY62l0LOEjxhbiIrqBAl1ADV8tszIaPd86m74A/LyuT20
# System services
services --enabled="chronyd"
# Do not configure the X Window System
skipx
# System timezone
timezone America/Chicago --isUtc
user --groups=wheel --name=hamzy --password=$6$!R}=iNm*FLyE)$0W7wfzyMzHT01HUIV0N4uH0Ixk.6n2AnsPMGr61zxraR0HZq.uS5uvkJDuxzvzyNo07712HNzQ2MAMv5EEnbQ1
EdgLFvZsSMbx1Ee8M1 --iscrypted
# System bootloader configuration
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=vda
autopart --type=lvm
# Partition clearing information
clearpart --all --initlabel --drives=vda

%packages
@core
chrony
kexec-tools
screen
tmux
wget
git

%end

%addon com_redhat_kdump --enable --reserve-mb='auto'

%end

%anaconda
pwpolicy root --minlen=6 --minquality=50 --notstrict --nochanges --notempty
pwpolicy user --minlen=6 --minquality=50 --notstrict --nochanges --notempty
pwpolicy luks --minlen=6 --minquality=50 --notstrict --nochanges --notempty
%end

%post --interpreter=/bin/bash --log=/root/anaconda-post.log
# Post configure tasks
set -x

hostnamectl set-hostname Overcloud.virbr0

SSH_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDB3tqPZ0+AAAAIw6G7XkYyRZecacl96Qocv9+prfN4Hs3bt68yLY7nlfhwYHDc+WFABvwq/jwylsjamznb4XV
s7ZWaiOMidYkAAAA6x+E/Fy4u8WGc/KVdYvhcDg2XvH96ja3nIAbVW9fGZ2rdrRkqqSLNLXC7IWfhsQbG/wF5+ddunBAAAApIuWS+xllgPQvd5O3iKRIqWPH6ZvR6dPx7gc8
a+RkjgqivgAAAARMfEtyQOwKxAHu9XX7w0ICTFOwuswyrRwOlW9piUnieHj6Yc/tpzpwmAAAAFTidQ/vmNdDJC/OKBifRYOeX54+RN7J2bPx66O1EcLdHLlh7c9qj hamzy@
hamzy-tp-w540"

for FILE in /root/.ssh/authorized_keys /home/hamzy/.ssh/authorized_keys
do
        mkdir -p $(dirname ${FILE})
        echo ${SSH_KEY} >> ${FILE}
        chmod 600 ${FILE}
        chmod 700 $(dirname ${FILE})
        if [[ ${FILE} == */root/* ]]
        then
                OWNER="root:root"
        elif [[ ${FILE} == */hamzy/* ]]
        then
                OWNER="hamzy:hamzy"
        fi
        chown ${OWNER} $(dirname ${FILE})
        chown ${OWNER} ${FILE}
done

echo "hamzy ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/hamzy

# Clean for epel-release install
yum clean all
yum update -y
# Install epel, then install pip separately
yum install -y epel-release
yum install -y python-pip
%end
__EOF__

Use virt-install to install CentOS7 inside a VM. 

[hamzy@pkvmci853 ~]$ sudo virsh destroy Overcloud; sudo virsh undefine Overcloud
Domain Overcloud destroyed
Domain Overcloud has been undefined
[hamzy@pkvmci853 ~]$ sudo virt-install --virt-type kvm --name Overcloud --memory 32768 --graphics none --disk ~/libvirt/Overcloud.qcow2,format=qcow2 --network=bridge:virbr0 --os-type=linux --os-variant=centos7.0 --location=http://mirror.centos.org/altarch/7/os/ppc64le/ --extra-args="inst.text console=tty0 console=ttyS0,115200 ks=file:/anaconda-ks-overcloud.cfg" --initrd-inject ~/libvirt/anaconda-ks-overcloud.cfg
...
        Installation complete.  Press return to quit
...
Domain creation completed.
Restarting guest.
...

Optionally, you can update the local hostname name for the VM. Libvirt configures dnsmasq to read the file /var/lib/libvirt/dnsmasq/default.addnhosts for hostnames. 

[hamzy@pkvmci853 ~]$ (FILE=/home/hamzy/libvirt/mac-dns.txt; > ${FILE}; for VMNAME in DLRN Overcloud; do MAC1=$(sudo virsh dumpxml ${VMNAME} | awk -F "'" '/mac address/ { print $2; exit }'); MAC2=${MAC1^^}; MAC3=${MAC2//:}; printf "${MAC3}\t${VMNAME}.virbr0\n" >> ${FILE}; done; cat ${FILE})
5254008E0DD2    DLRN.virbr0
525400CBFE22    Overcloud.virbr0
[hamzy@pkvmci853 ~]$ (FILE=/var/lib/libvirt/dnsmasq/default.addnhosts; sudo rm -f ${FILE}; sudo arp-scan --interface=virbr0 --macfile=/home/hamzy/libvirt/mac-dns.txt --localnet | head -n-3 | tail -n+3 | cut -f1,3- | sudo tee -a ${FILE})
192.168.122.104 DLRN.virbr0
192.168.122.147 Overcloud.virbr0

Install diskimage-builder

Diskimage-builder needs a hack patch to disable using python-greenlet from the CentOS repository. 

[hamzy@overcloud ~]$ git clone git://git.openstack.org/openstack/diskimage-builder
[hamzy@overcloud ~]$ (cd diskimage-builder; cat << '__EOF__' | git am -)
From b3bfe84ca40e70bce5609e274050e7e23bef00b9 Mon Sep 17 00:00:00 2001
From: Mark Hamzy <hamzy@us.ibm.com>
Date: Tue, 1 Aug 2017 12:44:12 -0500
Subject: [PATCH] fix python-greenlet conflict

---
 diskimage_builder/elements/yum/extra-data.d/99-yum-repo-conf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/diskimage_builder/elements/yum/extra-data.d/99-yum-repo-conf b/diskimage_builder/elements/yum/extra-data.d/99-yum-repo-conf
index b5b6e98..9fb27d0 100755
--- a/diskimage_builder/elements/yum/extra-data.d/99-yum-repo-conf
+++ b/diskimage_builder/elements/yum/extra-data.d/99-yum-repo-conf
@@ -25,3 +25,5 @@ for file in $DIB_YUM_REPO_CONF; do
     sudo cp -L -f $file $TMP_MOUNT_PATH/etc/yum.repos.d

 done
+
+echo 'exclude=python-greenlet*' | sudo tee -a $TMP_MOUNT_PATH/etc/yum.repos.d/CentOS-Base.repo
--
1.8.3.1

__EOF__
[hamzy@overcloud ~]$ (cd diskimage-builder/; sudo pip install --upgrade --force-reinstall --requirement requirements.txt)
[hamzy@overcloud ~]$ (cd diskimage-builder/; sudo python setup.py install --force)

Configure the repositories that the overcloud building process uses

[hamzy@overcloud ~]$ cat << __EOF__ | sudo tee -a /etc/yum.repos.d/DLRN.repo
[DLRN]
name=DLRN
baseurl=https://trunk.rdoproject.org/centos7/current-passed-ci/
enabled=1
gpgcheck=0
cost=1
__EOF__
[hamzy@overcloud ~]$ cat << __EOF__ | sudo tee -a /etc/yum.repos.d/cloud7-openstack-common-candidate.repo
[cloud7_openstack_common_candidate]
name=cloud7_openstack_common_candidate
baseurl=https://cbs.centos.org/repos/cloud7-openstack-common-candidate/ppc64le/os/
enabled=1
gpgcheck=0
cost=2
__EOF__
[hamzy@overcloud ~]$ cat << __EOF__ | sudo tee -a /etc/yum.repos.d/cloud7-openstack-common-testing.repo
[cloud7_openstack_common_testing]
name=cloud7_openstack_common_testing
baseurl=https://cbs.centos.org/repos/cloud7-openstack-common-testing/ppc64le/os/
enabled=1
gpgcheck=0
cost=3
__EOF__
[hamzy@overcloud ~]$ cat << __EOF__ | sudo tee -a /etc/yum.repos.d/cloud7-openstack-common-release.repo
[cloud7_openstack_common_release]
name=cloud7_openstack_common_release
baseurl=https://cbs.centos.org/repos/cloud7-openstack-common-release/ppc64le/os/
enabled=1
gpgcheck=0
cost=4
__EOF__
[hamzy@overcloud ~]$ cat << __EOF__ | sudo tee -a /etc/yum.repos.d/cloud7-openstack-pike-testing.repo
[cloud7_openstack_pike_testing]
name=cloud7_openstack_pike_testing
baseurl=http://cbs.centos.org/repos/cloud7-openstack-pike-testing/ppc64le/os/
enabled=1
gpgcheck=0
cost=5
__EOF__
[hamzy@overcloud ~]$ sudo yum clean all; sudo yum update -y

erlang-sd_notify is broken

Erlang is broken for ppc64le. You will see the following during the build: 

...
Error: Package: erlang-sd_notify-0.1-9.el7.ppc64le (cloud7_openstack_common_candidate)
           Requires: erlang(erl_nif_version) = 2.11
           Installing: erlang-erts-18.3.4.5-4.el7.ppc64le (cloud7_openstack_common_candidate)
               erlang(erl_nif_version) = 2.10
...

So, to fix it, you need to add a missing repository on a web server: 

[hamzy@pkvmci853 ~]$ (DIR=/var/www/html/missing-overcloud; cd ${DIR}; sudo cbs download-build --arch=ppc64le --arch=noarch erlang-19.3.6.1-1.el7 --quiet; sudo createrepo .; sudo chcon -R system_u:object_r:httpd_sys_content_t:s0 ${DIR})

And add another repository to the overcloud builder: 

[hamzy@overcloud ~]$ cat << __EOF__ | sudo tee -a /etc/yum.repos.d/missing-overcloud.repo
[missing_overcloud]
name=missing_overcloud
baseurl=http://192.168.122.1/missing-overcloud/
enabled=1
gpgcheck=0
cost=6
__EOF__

Install python-tripleoclient

Install python-tripleoclient and patch the overcloud-images.yaml file to use ppc64le instead of amd64. 

[hamzy@overcloud ~]$ sudo yum install -y python-tripleoclient

Prepare for local patches

[hamzy@overcloud ~]$ (cd /usr/share/openstack-tripleo-common/; sudo git init .; sudo find . | xargs sudo git add; sudo git commit -m "base")
[hamzy@overcloud ~]$ (cd /usr/lib/python2.7/site-packages/tripleo_common/; sudo git init .; sudo find . -not -iname \*.pyc -and -not -iname \*.pyo | xargs sudo git add; sudo git commit -m "base")

Install patches to allow building of ppc64le images

If https://review.openstack.org/#/c/489476/ has not been accepted, then pull it in. 

[hamzy@overcloud ~]$ (cd /usr/share/openstack-tripleo-common; wget --quiet -O - 'https://review.openstack.org/changes/489476/revisions/fba2902dc6aacffd04697dc72dfd5e5b8a427eec/patch?download' | base64 --decode | awk '/^diff --git a\/releasenotes\/notes\/default-arch-selection-d5fd2fcdba725dd4/{found=1;}
{if (!found) {print;}}' | sudo patch -p1)
patching file image-yaml/overcloud-hardened-images-centos7.yaml
patching file image-yaml/overcloud-hardened-images-rhel7.yaml
patching file image-yaml/overcloud-hardened-images.yaml
patching file image-yaml/overcloud-images-centos7.yaml
patching file image-yaml/overcloud-images-rhel7.yaml
patching file image-yaml/overcloud-images.yaml
patching file image-yaml/overcloud-odl-rhel7.yaml
[hamzy@overcloud ~]$ (cd /usr/lib/python2.7/site-packages/tripleo_common/; wget --quiet -O - 'https://review.openstack.org/changes/489476/revisions/fba2902dc6aacffd04697dc72dfd5e5b8a427eec/patch?download' | base64 --decode | awk '/^diff --git a\/tripleo_common\/arch.py/{i++}i' | sudo patch -p2)
patching file arch.py
patching file image/build.py
patching file tests/test_arch.py

Install patch to build a full disk image

[hamzy@overcloud ~]$ (cd /usr/share/openstack-tripleo-common/; cat << '__EOF__' | sudo git am)
From 1e0e7d61247a6ba64786e5be59ec4c7e7a359552 Mon Sep 17 00:00:00 2001
From: root <root@overcloud.virbr0>
Date: Sun, 5 Nov 2017 08:49:34 -0600
Subject: [PATCH] openstack-tripleo-common overcloud-images

---
 image-yaml/overcloud-images.yaml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/image-yaml/overcloud-images.yaml b/image-yaml/overcloud-images.yaml
index c3a0312..52783f9 100644
--- a/image-yaml/overcloud-images.yaml
+++ b/image-yaml/overcloud-images.yaml
@@ -5,6 +5,8 @@ disk_images:
     elements:
       - baremetal
       - dhcp-all-interfaces
+      - vm
+      - devuser
       - overcloud-agent
       - overcloud-full
       - overcloud-controller
-- 
1.8.3.1
__EOF__
Applying: openstack-tripleo-common overcloud-images

Modifying overcloud-full.qcow2 to have a prep partition

[hamzy@overcloud ~]$ cat << '__EOF__' > part.yaml
  - local_loop:
      name: image0

  - partitioning:
      base: image0
      label: mbr
      partitions:
        - name:  boot
          flags: [ boot, primary ]
          size:  4MiB
          type:  "0x41"
        - name:  root
          flags: [ primary ]
          size:  100%
          mkfs:
            label: markfs
            type:  ext4
            mount:
              mount_point: /
              fstab:
                options: "defaults"
                fsck-passno: 1
__EOF__
[hamzy@overcloud ~]$ export DIB_BLOCK_DEVICE_CONFIG=$(cat part.yaml)
[hamzy@overcloud ~]$ sudo /bin/rm -rf overcloud-full*

Run the overcloud building command

[hamzy@overcloud ~]$ (export DIB_DEV_USER_PWDLESS_SUDO=yes; export DIB_DEV_USER_AUTHORIZED_KEYS=/home/ubuntu/hamzy_id_rsa.pub; export DIB_DEV_USER_USERNAME=hamzy; export DIB_DEV_USER_PASSWORD=password; export DIB_YUM_REPO_CONF="/etc/yum.repos.d/DLRN.repo /etc/yum.repos.d/cloud7-openstack-common-candidate.repo /etc/yum.repos.d/cloud7-openstack-common-testing.repo /etc/yum.repos.d/cloud7-openstack-common-release.repo /etc/yum.repos.d/cloud7-openstack-pike-candidate.repo /etc/yum.repos.d/cloud7-openstack-pike-testing.repo /etc/yum.repos.d/missing-overcloud.repo"; export NODE_ARCH=ppc64le; export DIB_CLOUD_IMAGES="http://ftp.unicamp.br/pub/ppc64el/centos/7/images/"; export BASE_IMAGE_FILE="CentOS-7-ppc64le-GenericCloud-1611.qcow2"; time openstack overcloud image build) 2>&1 | tee output.overcloud-image-build)

(Optionally) modify IPA initramfs to have a password for root

[hamzy@overcloud ~]$ cp ironic-python-agent.initramfs ironic-python-agent.initramfs.orig
[hamzy@overcloud ~]$ mv ironic-python-agent.initramfs ironic-python-agent.initramfs.gz
[hamzy@overcloud ~]$ gzip --decompress --to-stdout ironic-python-agent.initramfs.orig > ironic-python-agent.initramfs.cpio
[hamzy@overcloud ~]$ mkdir ironic-python-agent; cd ironic-python-agent; sudo cpio -id --no-preserve-owner --preserve-modification-time < ~/ironic-python-agent.initramfs.cpio)
[hamzy@overcloud ~]$ (PASS=$(openssl passwd -1 -salt xyz password); sudo sed -i -e 's,^root:!!,root:'${PASS}',' ironic-python-agent/etc/shadow)
[hamzy@overcloud ~]$ (cd ironic-python-agent/; sudo find . | sudo cpio -H newc -o > ~/ironic-python-agent.initramfs.cpio)
[hamzy@overcloud ~]$ gzip < ~/ironic-python-agent.initramfs.cpio > ~/ironic-python-agent.initramfs.gz
[hamzy@overcloud ~]$ cp ironic-python-agent.initramfs.gz ironic-python-agent.initramfs
Retrieved from "https://fedoraproject.org/w/index.php?title=User:Hamzy/ppc64le_Overcloud_builder_instance&oldid=505053"