From Fedora Project Wiki
On a CentOS7 system
[hamzy@oscloud5 ~]$ lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.3.1611 (Core) Release: 7.3.1611 Codename: Core [stack@oscloud5 ~]$ uname -a Linux oscloud5.stglabs.ibm.com 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Unfortunately, it seems that Environment setup for baremetal environment does not explain how to install the undercloud. There are three machines in this scenario:
| arch | use | portname1 | MAC1 | IP1 | portname2 | MAC2 | IP2 |
| x86_64 | undercloud | eno2 | 6c:ae:8b:29:2a:02 | 9.114.219.30 | eno4 | 6c:ae:8b:29:2a:04 | 9.114.118.98 |
| ppc64le | overcloud control | enP3p9s0f0 | 6c:ae:8b:6a:74:14 | 9.114.219.134 | enp1s0 | 34:40:b5:b6:ea:bc | 9.114.118.50 |
| ppc64le | overcloud compute | enP3p5s0f2 | 00:90:fa:74:05:52 | 9.114.219.49 | enP3p5s0f3 | 00:90:fa:74:05:53 | 9.114.118.154 |
So, following Undercloud installation, I perform the following:
[hamzy@oscloud5 ~]$ sudo useradd stack
[hamzy@oscloud5 ~]$ sudo passwd stack
[hamzy@oscloud5 ~]$ echo "stack ALL=(root) NOPASSWD:ALL" | sudo tee -a /etc/sudoers.d/stack
[hamzy@oscloud5 ~]$ sudo chmod 0440 /etc/sudoers.d/stack
[hamzy@oscloud5 ~]$ sudo su - stack
[stack@oscloud5 ~]$ sudo hostnamectl set-hostname oscloud5.stglabs.ibm.com
[stack@oscloud5 ~]$ sudo hostnamectl set-hostname --transient oscloud5.stglabs.ibm.com
[stack@oscloud5 ~]$ sudo curl -L -o /etc/yum.repos.d/delorean.repo https://trunk.rdoproject.org/centos7-master/current-passed-ci/delorean.repo
[stack@oscloud5 ~]$ sudo curl -L -o /etc/yum.repos.d/delorean-deps.repo https://trunk.rdoproject.org/centos7/delorean-deps.repo
[stack@oscloud5 ~]$ sudo yum install -y python-tripleoclient
[stack@oscloud5 ~]$ cp /usr/share/instack-undercloud/undercloud.conf.sample ~/undercloud.conf
[stack@oscloud5 ~]$ cat << '__EOF__' > instackenv.json
{
"nodes": [
{
"pm_type":"pxe_ipmitool",
"mac":[
"34:40:b5:b6:ea:bc"
],
"cpu":"16",
"memory":"1048576",
"disk":"1000",
"arch":"ppc64le",
"pm_password":"update",
"pm_addr":"9.114.118.51"
},
{
"pm_type":"pxe_ipmitool",
"mac":[
"00:90:fa:74:05:53"
],
"cpu":"16",
"memory":"1048576",
"disk":"1000",
"arch":"ppc64le",
"pm_password":"update",
"pm_addr":"9.114.118.155"
}
]
}
__EOF__
I transfer over the built overcloud images:
[hamzy@pkvmci853 ~]$ (OCB=$(dig @192.168.122.1 -4 +short Overcloud.virbr0); UC=9.114.118.98; ssh-keygen -f ~/.ssh/known_hosts -R ${UC}; ssh-keyscan ${UC} >> ~/.ssh/known_hosts; scp -3 hamzy@${OCB}:~/*{initrd,initramfs,kernel,vmlinuz,qcow2}* stack@${UC}:~/)
I then modify undercloud.conf as follows:
[stack@oscloud5 ~]$ cat << __EOF__ | patch -p0 --- undercloud.conf.orig 2017-08-25 12:04:54.935063830 +0000 +++ undercloud.conf 2017-08-25 12:05:17.561063576 +0000 @@ -17,21 +17,25 @@ # defined by local_interface, with the netmask defined by the prefix # portion of the value. (string value) #local_ip = 192.168.24.1/24 +local_ip = 9.114.118.98/24 # Network gateway for the Neutron-managed network for Overcloud # instances. This should match the local_ip above when using # masquerading. (string value) #network_gateway = 192.168.24.1 +network_gateway = 9.114.118.98 # Virtual IP or DNS address to use for the public endpoints of # Undercloud services. Only used with SSL. (string value) # Deprecated group/name - [DEFAULT]/undercloud_public_vip #undercloud_public_host = 192.168.24.2 +undercloud_public_host = 9.114.118.98 # Virtual IP or DNS address to use for the admin endpoints of # Undercloud services. Only used with SSL. (string value) # Deprecated group/name - [DEFAULT]/undercloud_admin_vip #undercloud_admin_host = 192.168.24.3 +undercloud_admin_host = 9.114.118.98 # DNS nameserver(s) to use for the undercloud node. (list value) #undercloud_nameservers = @@ -74,6 +78,7 @@ # Network interface on the Undercloud that will be handling the PXE # boots and DHCP for Overcloud instances. (string value) #local_interface = eth1 +local_interface = eno4 # MTU to use for the local_interface. (integer value) #local_mtu = 1500 @@ -82,18 +87,22 @@ # instances. This should be the subnet used for PXE booting. (string # value) #network_cidr = 192.168.24.0/24 +network_cidr = 9.114.118.0/24 # Network that will be masqueraded for external access, if required. # This should be the subnet used for PXE booting. (string value) #masquerade_network = 192.168.24.0/24 +masquerade_network = 9.114.118.0/24 # Start of DHCP allocation range for PXE and DHCP of Overcloud # instances. (string value) #dhcp_start = 192.168.24.5 +dhcp_start = 9.114.118.240 # End of DHCP allocation range for PXE and DHCP of Overcloud # instances. (string value) #dhcp_end = 192.168.24.24 +dhcp_end = 9.114.118.248 # Path to hieradata override file. If set, the file will be copied # under /etc/puppet/hieradata and set as the first file in the hiera @@ -112,12 +121,14 @@ # doubt, use the default value. (string value) # Deprecated group/name - [DEFAULT]/discovery_interface #inspection_interface = br-ctlplane +inspection_interface = br-ctlplane # Temporary IP range that will be given to nodes during the inspection # process. Should not overlap with the range defined by dhcp_start # and dhcp_end, but should be in the same network. (string value) # Deprecated group/name - [DEFAULT]/discovery_iprange #inspection_iprange = 192.168.24.100,192.168.24.120 +inspection_iprange = 9.114.118.249,9.114.118.250 # Whether to enable extra hardware collection during the inspection # process. Requires python-hardware or python-hardware-detect package __EOF__
And install the undercloud:
[stack@oscloud5 ~]$ time openstack undercloud install 2>&1 | tee output.undercloud.install ... Undercloud install complete. ...
There is a bug for needing the userid for machines using ipmi that needs to be patched around.
[stack@oscloud5 ~]$ (cd /usr/lib/python2.7/site-packages/tripleo_common/utils/; cat << __EOF__ | sudo patch -p0)
--- nodes.py.orig 2017-08-24 15:54:07.614226329 +0000
+++ nodes.py 2017-08-24 15:54:29.699440619 +0000
@@ -105,7 +105,7 @@
'pm_user': '%s_username' % prefix,
'pm_password': '%s_password' % prefix,
}
- mandatory_fields = list(mapping)
+ mandatory_fields = ['pm_addr', 'pm_password'] # list(mapping)
if has_port:
mapping['pm_port'] = '%s_port' % prefix
__EOF__
[stack@undercloud ~]$ (for SERVICE in openstack-mistral-api.service openstack-mistral-engine.service openstack-mistral-executor.service; do sudo systemctl restart ${SERVICE}; done)
Ironic needs some different settings to be able to support PXE for ppc64le:
[stack@oscloud5 ~]$ (cd /etc/ironic; cat << '__EOF__' | sudo patch -p0)
--- ironic.conf.orig 2017-09-11 17:46:28.760794196 +0000
+++ ironic.conf 2017-09-11 17:49:55.637796731 +0000
@@ -343,6 +343,7 @@
# for this option to be unset. (string value)
# Allowed values: debug, info, warning, error, critical
#notification_level = <None>
+notification_level = debug
# Directory where the ironic python module is installed.
# (string value)
@@ -3512,6 +3513,7 @@
# configuration per node architecture. For example:
# aarch64:/opt/share/grubaa64_pxe_config.template (dict value)
#pxe_config_template_by_arch =
+pxe_config_template_by_arch = ppc64le:$pybasedir/drivers/modules/pxe_config.template
# IP address of ironic-conductor node's TFTP server. (string
# value)
@@ -3551,10 +3553,11 @@
# Bootfile DHCP parameter per node architecture. For example:
# aarch64:grubaa64.efi (dict value)
#pxe_bootfile_name_by_arch =
+pxe_bootfile_name_by_arch = ppc64le:config
# Enable iPXE boot. (boolean value)
#ipxe_enabled = false
-ipxe_enabled=True
+ipxe_enabled = false
# On ironic-conductor node, the path to the main iPXE script
# file. (string value)
__EOF__
[stack@oscloud5 ~]$ for I in openstack-ironic-conductor.service openstack-ironic-inspector.service openstack-ironic-inspector-dnsmasq.service; do sudo systemctl restart ${I}; done
I then go through the process of installing the overcloud:
[stack@oscloud5 ~]$ source stackrc (undercloud) [stack@oscloud5 ~]$ time openstack overcloud image upload ...
The overcloud-full qcow2 image needs to be recreated in glance so that it loses both the kernel_id and the ramdisk_id. This way a full disk image can be deployed.
(undercloud) [stack@oscloud5 ~]$ (FILE="overcloud-full.qcow2"; UUID=$(openstack image list -f value | grep 'overcloud-full ' | awk '{print $1;}'); openstack image delete ${UUID}; openstack image create --container-format bare --disk-format qcow2 --min-disk 0 --min-ram 0 --file ${FILE} --public overcloud-full)
Now import the baremetal nodes and assign them profiles.
(undercloud) [stack@oscloud5 ~]$ time openstack overcloud node import --provide instackenv.json 2>&1 | tee output.overcloud.node.import
...
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
| Node UUID | Node Name | Provision State | Current Profile | Possible Profiles |
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
| ff2fdac5-6cc5-47a9-a095-d942b3960795 | | available | None | |
| ef3d7b3b-97b8-42ab-b501-896474df658f | | available | None | |
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
(undercloud) [stack@oscloud5 ~]$ (COMPUTE=""; CONTROL=""; while IFS=$' ' read -r -a PROFILES; do if [ -z "${COMPUTE}" ]; then COMPUTE=${PROFILES[0]}; ironic node-update ${COMPUTE} replace properties/capabilities=profile:compute,boot_option:local; continue; fi; if [ -z "${CONTROL}" ]; then CONTROL=${PROFILES[0]}; ironic node-update ${CONTROL} replace properties/capabilities=profile:control,boot_option:local; continue; fi; done < <(openstack overcloud profiles list -f value))
(undercloud) [stack@oscloud5 ~]$ openstack overcloud profiles list
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
| Node UUID | Node Name | Provision State | Current Profile | Possible Profiles |
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
| ff2fdac5-6cc5-47a9-a095-d942b3960795 | | available | compute | |
| ef3d7b3b-97b8-42ab-b501-896474df658f | | available | control | |
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
And now do the deploy.
(undercloud) [stack@oscloud5 ~]$ cp -r /usr/share/openstack-tripleo-heat-templates templates (undercloud) [stack@oscloud5 ~]$ wget --quiet -O templates/diff.patch https://paste.fedoraproject.org/paste/lfB5Bd-hRBCmHxS5SaiyvA/raw # http://paste.openstack.org/raw/621668/ (no eol bug) (undercloud) [stack@oscloud5 ~]$ (cd templates/; patch -p 1 < diff.patch) (undercloud) [stack@oscloud5 ~]$ time openstack overcloud deploy --debug --templates /home/stack/templates -e /home/stack/templates/environments/network-environment.yaml -e /home/stack/templates/environments/network-isolation-custom.yaml --control-scale 1 --compute-scale 1 --control-flavor control --compute-flavor compute 2>&1 | tee output.overcloud.deploy
However, if you see the following error:
(undercloud) [stack@oscloud5 ~]$ openstack overcloud deploy --debug --templates -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml --control-scale 1 --compute-scale 1 --control-flavor control --compute-flavor compute 2>&1 | tee output.overcloud.deploy
...
(undercloud) [stack@oscloud5 ~]$ sudo cat /var/log/ironic/ironic-conductor.log
...
/var/log/ironic/ironic-conductor.log:2017-09-01 14:16:03.633 5119 ERROR ironic.drivers.modules.agent_base_vendor [req-dd8d4e4f-91cd-
4d5b-b41a-6969d43a9495 - - - - -] Asynchronous exception for node 58c6e4a4-b0ec-49a7-9b5a-368a88c50dc3: Node failed to deploy. Excep
tion: Failed to deploy instance: Failed to start the iSCSI target to deploy the node 58c6e4a4-b0ec-49a7-9b5a-368a88c50dc3. Error: {u
'message': u"'exceptions.OSError' object has no attribute 'exit_code'", u'code': 500, u'type': u'AttributeError', u'details': u''}:
InstanceDeployFailure: Failed to deploy instance: Failed to start the iSCSI target to deploy the node 58c6e4a4-b0ec-49a7-9b5a-368a88
c50dc3. Error: {u'message': u"'exceptions.OSError' object has no attribute 'exit_code'", u'code': 500, u'type': u'AttributeError', u
'details': u''}
...
then this is because the scsi-target-utils package does not seem to be installed.
(undercloud) [stack@oscloud5 ~]$ cp ironic-python-agent.initramfs ironic-python-agent.initramfs.orig (undercloud) [stack@oscloud5 ~]$ mv ironic-python-agent.initramfs ironic-python-agent.initramfs.gz (undercloud) [stack@oscloud5 ~]$ gzip --decompress --to-stdout ironic-python-agent.initramfs.gz > ironic-python-agent.initramfs.cpio (undercloud) [stack@oscloud5 ~]$ file ironic-python-agent.initramfs.cpio ironic-python-agent.initramfs.cpio: ASCII cpio archive (SVR4 with no CRC) (undercloud) [stack@oscloud5 ~]$ (mkdir ironic-python-agent; cd ironic-python-agent; cpio -id < ~/ironic-python-agent.initramfs.cpio) (undercloud) [stack@oscloud5 ~]$ find ironic-python-agent/ -iname \*tgt\* ironic-python-agent/usr/lib/modules/3.10.0-514.26.2.el7.ppc64le/kernel/drivers/scsi/ibmvscsi/ibmvstgt.ko ironic-python-agent/usr/lib/modules/3.10.0-514.26.2.el7.ppc64le/kernel/drivers/scsi/scsi_tgt.ko ironic-python-agent/etc/selinux/targeted/active/modules/100/tgtd
There is also a bug [1] written which should help expose the real error.
However, using a properly build overcloud-full image, you will see the following error:
(undercloud) [stack@oscloud5 ~]$ sudo cat /var/log/ironic/ironic-conductor.log ... 2017-09-10 22:49:38.857 27228 ERROR ironic.drivers.modules.iscsi_deploy Couldn't determine the UUID of the root partition or the disk identifier after deploying node 032a8e33-e371-44e3-8513-04028a4de95b ...
Unfortunately there are x86_64 isms that do not work under ppc64le.