User:Hamzy/TripleO mixed undercloud overcloud try9

On a CentOS7 system

[hamzy@oscloud5 ~]$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.3.1611 (Core) 
Release:        7.3.1611
Codename:       Core
[stack@oscloud5 ~]$ uname -a
Linux oscloud5.stglabs.ibm.com 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Unfortunately, it seems that Environment setup for baremetal environment does not explain how to install the undercloud. There are three machines in this scenario:

arch	use	portname1	MAC1	IP1	portname2	MAC2	IP2
x86_64	undercloud	eno2	6c:ae:8b:29:2a:02	9.114.219.30	eno4	6c:ae:8b:29:2a:04	9.114.118.98
x86_64	overcloud control	eth???	00:0a:f7:73:3c:c3	9.114.219.134	eth???	6c:ae:8b:29:27:fa	9.114.118.??? 13?
ppc64le	overcloud compute	enP3p5s0f2	00:90:fa:74:05:52	9.114.219.49	enP3p5s0f3	00:90:fa:74:05:53	9.114.118.154

So, following Undercloud installation, I perform the following:

[hamzy@oscloud5 ~]$ sudo useradd stack
[hamzy@oscloud5 ~]$ sudo passwd stack
[hamzy@oscloud5 ~]$ echo "stack ALL=(root) NOPASSWD:ALL" | sudo tee -a /etc/sudoers.d/stack
[hamzy@oscloud5 ~]$ sudo chmod 0440 /etc/sudoers.d/stack
[hamzy@oscloud5 ~]$ sudo su - stack
[stack@oscloud5 ~]$ sudo hostnamectl set-hostname oscloud5.stglabs.ibm.com
[stack@oscloud5 ~]$ sudo hostnamectl set-hostname --transient oscloud5.stglabs.ibm.com
[stack@oscloud5 ~]$ sudo curl -L -o /etc/yum.repos.d/delorean.repo https://trunk.rdoproject.org/centos7-master/current-passed-ci/delorean.repo
[stack@oscloud5 ~]$ sudo curl -L -o /etc/yum.repos.d/delorean-deps.repo https://trunk.rdoproject.org/centos7/delorean-deps.repo
[stack@oscloud5 ~]$ sudo yum install -y python-tripleoclient
[stack@oscloud5 ~]$ cp /usr/share/instack-undercloud/undercloud.conf.sample ~/undercloud.conf
[stack@oscloud5 ~]$ cat << '__EOF__' > instackenv.json
{
    "nodes": [
        {
            "pm_type":"pxe_ipmitool",
            "mac":[
                "5c:f3:fc:36:dd:68"
            ],
            "cpu":"2",
            "memory":"1048576",
            "disk":"1000",
            "arch":"x86_64",
            "pm_user":"user",
            "pm_password":"update",
            "pm_addr":"9.114.219.33"
        },
        {
            "pm_type":"pxe_ipmitool",
            "mac":[
                "00:90:fa:74:05:53"
            ],
            "cpu":"16",
            "memory":"1048576",
            "disk":"1000",
            "arch":"ppc64le",
            "pm_password":"update",
            "pm_addr":"9.114.118.155"
        }
    ]
}
__EOF__

I transfer over the built ppc64le overcloud images:

[hamzy@pkvmci853 ~]$ (OCB=$(dig @192.168.122.1 -4 +short Overcloud.virbr0); UC=9.114.118.98; ssh-keygen -f ~/.ssh/known_hosts -R ${UC}; ssh-keyscan ${UC} >> ~/.ssh/known_hosts; scp -3 hamzy@${OCB}:~/ironic-python-agent.initramfs stack@${UC}:~/ppc64le-bm-deploy-ramdisk; scp -3 hamzy@${OCB}:~/ironic-python-agent.kernel stack@${UC}:~/ppc64le-bm-deploy-kernel; scp -3 hamzy@${OCB}:~/overcloud-full.qcow2 stack@${UC}:~/ppc64le-overcloud-full.qcow2; scp -3 hamzy@${OCB}:~/overcloud-full.initrd stack@${UC}:~/ppc64le-overcloud-full.initrd; scp -3 hamzy@${OCB}:~/overcloud-full.kernel stack@${UC}:~/ppc64le-overcloud-full.kernel)

I then transfer over the built x86_64 overcloud images:

[stack@oscloud5 ~]$ wget --quiet -O - https://images.rdoproject.org/pike/rdo_trunk/d558d6173970736fde63526da6824d391ada6042_89d45961/ironic-python-agent.tar | tar xvf -
[stack@oscloud5 ~]$ wget --quiet -O - https://images.rdoproject.org/pike/rdo_trunk/d558d6173970736fde63526da6824d391ada6042_89d45961/overcloud-full.tar | tar xvf -
[stack@oscloud5 ~]$ for FILE in ironic-python-agent.initramfs ironic-python-agent.kernel overcloud-full.initrd overcloud-full.vmlinuz overcloud-full.qcow2; do mv ${FILE} x86_64-${FILE}; done

I then modify undercloud.conf as follows:

[stack@oscloud5 ~]$ cat << __EOF__ | patch -p0
--- undercloud.conf.orig        2017-08-25 12:04:54.935063830 +0000
+++ undercloud.conf 2017-08-25 12:05:17.561063576 +0000
@@ -17,21 +17,25 @@
 # defined by local_interface, with the netmask defined by the prefix
 # portion of the value. (string value)
 #local_ip = 192.168.24.1/24
+local_ip = 9.114.118.98/24
 
 # Network gateway for the Neutron-managed network for Overcloud
 # instances. This should match the local_ip above when using
 # masquerading. (string value)
 #network_gateway = 192.168.24.1
+network_gateway = 9.114.118.98
 
 # Virtual IP or DNS address to use for the public endpoints of
 # Undercloud services. Only used with SSL. (string value)
 # Deprecated group/name - [DEFAULT]/undercloud_public_vip
 #undercloud_public_host = 192.168.24.2
+undercloud_public_host = 9.114.118.98
 
 # Virtual IP or DNS address to use for the admin endpoints of
 # Undercloud services. Only used with SSL. (string value)
 # Deprecated group/name - [DEFAULT]/undercloud_admin_vip
 #undercloud_admin_host = 192.168.24.3
+undercloud_admin_host = 9.114.118.98
 
 # DNS nameserver(s) to use for the undercloud node. (list value)
 #undercloud_nameservers =
@@ -74,6 +78,7 @@
 # Network interface on the Undercloud that will be handling the PXE
 # boots and DHCP for Overcloud instances. (string value)
 #local_interface = eth1
+local_interface = eno4
 
 # MTU to use for the local_interface. (integer value)
 #local_mtu = 1500
@@ -82,18 +87,22 @@
 # instances. This should be the subnet used for PXE booting. (string
 # value)
 #network_cidr = 192.168.24.0/24
+network_cidr = 9.114.118.0/24
 
 # Network that will be masqueraded for external access, if required.
 # This should be the subnet used for PXE booting. (string value)
 #masquerade_network = 192.168.24.0/24
+masquerade_network = 9.114.118.0/24
 
 # Start of DHCP allocation range for PXE and DHCP of Overcloud
 # instances. (string value)
 #dhcp_start = 192.168.24.5
+dhcp_start = 9.114.118.240
 
 # End of DHCP allocation range for PXE and DHCP of Overcloud
 # instances. (string value)
 #dhcp_end = 192.168.24.24
+dhcp_end = 9.114.118.248
 
 # Path to hieradata override file. If set, the file will be copied
 # under /etc/puppet/hieradata and set as the first file in the hiera
@@ -112,12 +121,14 @@
 # doubt, use the default value. (string value)
 # Deprecated group/name - [DEFAULT]/discovery_interface
 #inspection_interface = br-ctlplane
+inspection_interface = br-ctlplane
 
 # Temporary IP range that will be given to nodes during the inspection
 # process.  Should not overlap with the range defined by dhcp_start
 # and dhcp_end, but should be in the same network. (string value)
 # Deprecated group/name - [DEFAULT]/discovery_iprange
 #inspection_iprange = 192.168.24.100,192.168.24.120
+inspection_iprange = 9.114.118.249,9.114.118.250
 
 # Whether to enable extra hardware collection during the inspection
 # process. Requires python-hardware or python-hardware-detect package
__EOF__

And install the undercloud:

[stack@oscloud5 ~]$ time openstack undercloud install 2>&1 | tee output.undercloud.install
...
Undercloud install complete.
...

Prepare your system for some patches...

[stack@oscloud5 ~]$ (cd /usr/share/openstack-tripleo-common/; sudo git init .; sudo find . | xargs sudo git add; sudo git commit -m "base")
[stack@oscloud5 ~]$ (cd /usr/lib/python2.7/site-packages/tripleoclient/; sudo git init .; sudo find . -not -iname \*.pyc -and -not -iname \*.pyo | xargs sudo git add; sudo git commit -m "base")
[stack@oscloud5 ~]$ (cd /usr/lib/python2.7/site-packages/tripleo_common/; sudo git init .; sudo find . -not -iname \*.pyc -and -not -iname \*.pyo | xargs sudo git add; sudo git commit -m "base")

There is a bug for needing the userid for machines using ipmi that needs to be patched around.

[stack@oscloud5 ~]$ (cd /usr/lib/python2.7/site-packages/tripleo_common/; cat << '__EOF__' | sudo git am)
From ceb4796a442990dc99917a83a2c6bf502fe71fbf Mon Sep 17 00:00:00 2001
From: root <root@oscloud5.stglabs.ibm.com>
Date: Fri, 3 Nov 2017 15:31:41 +0000
Subject: [PATCH] fix usernameless IPMI

---
 utils/nodes.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/utils/nodes.py b/utils/nodes.py
index d160a00..0d93804 100644
--- a/utils/nodes.py
+++ b/utils/nodes.py
@@ -105,7 +105,7 @@ class PrefixedDriverInfo(DriverInfo):
             'pm_user': '%s_username' % prefix,
             'pm_password': '%s_password' % prefix,
         }
-        mandatory_fields = list(mapping)
+        mandatory_fields = ['pm_addr', 'pm_password'] # list(mapping)
 
         if has_port:
             mapping['pm_port'] = '%s_port' % prefix
-- 
1.8.3.1
__EOF__

There is a bug for hardcoded ironic-python-agent and overcloud names:

[stack@oscloud5 ~]$ (cd /usr/share/openstack-tripleo-common/; cat << '__EOF__' | sudo git am)
From 139ca1ef0fc589da608edde67f66c4b398b7bff0 Mon Sep 17 00:00:00 2001
From: root <root@oscloud5.stglabs.ibm.com>
Date: Wed, 1 Nov 2017 16:52:00 +0000
Subject: [PATCH] openstack-tripleo-common

---
 workbooks/baremetal.yaml   |  4 ----
 workbooks/validations.yaml | 13 ++++++++-----
 2 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/workbooks/baremetal.yaml b/workbooks/baremetal.yaml
index 90bca8a..e1f978d 100644
--- a/workbooks/baremetal.yaml
+++ b/workbooks/baremetal.yaml
@@ -147,8 +147,6 @@ workflows:
      - nodes_json
      - remove: False
      - queue_name: tripleo
-     - kernel_name: null
-     - ramdisk_name: null
      - instance_boot_option: local
      - initial_state: manageable

@@ -181,8 +179,6 @@ workflows:
         input:
            nodes_json: <% $.nodes_json %>
            remove: <% $.remove %>
-           kernel_name: <% $.kernel_name %>
-           ramdisk_name: <% $.ramdisk_name %>
            instance_boot_option: <% $.instance_boot_option %>
         publish:
           registered_nodes: <% task(register_or_update_nodes).result %>
diff --git a/workbooks/validations.yaml b/workbooks/validations.yaml
index 5cb9ad7..1799ee5 100644
--- a/workbooks/validations.yaml
+++ b/workbooks/validations.yaml
@@ -400,8 +400,6 @@ workflows:

   check_ironic_boot_configuration:
     input:
-      - kernel_id: null
-      - ramdisk_id: null
       - run_validations: true
       - queue_name: tripleo
     output:
@@ -419,7 +417,7 @@ workflows:

       get_ironic_nodes:
         action: ironic.node_list
-        on-success: check_node_boot_configuration
+        on-success: get_images
         on-error: failed_get_ironic_nodes
         input:
           provision_state: available
@@ -434,12 +432,17 @@ workflows:
           status: FAILED
           message: <% task(get_ironic_nodes).result %>

+      get_images:
+        action: glance.images_list
+        on-success: check_node_boot_configuration
+        publish:
+          images: <% task(get_images).result %>
+
       check_node_boot_configuration:
         action: tripleo.validations.check_node_boot_configuration
         input:
           node: <% $.node %>
-          kernel_id: <% $.kernel_id %>
-          ramdisk_id: <% $.ramdisk_id %>
+          images: <% $.images %>
         with-items: node in <% $.nodes %>
         on-success: send_message
         on-error: fail_check_node_boot_configuration
-- 
1.8.3.1
__EOF__

[stack@oscloud5 ~]$ (cd /usr/lib/python2.7/site-packages/tripleoclient/; cat << '__EOF__' | sudo git am)
From 475ed6c45a9f63b6596ce155f4aeb19ad8e135a0 Mon Sep 17 00:00:00 2001
From: root <root@oscloud5.stglabs.ibm.com>
Date: Wed, 1 Nov 2017 16:56:13 +0000
Subject: [PATCH] tripleoclient

---
 v1/overcloud_node.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/v1/overcloud_node.py b/v1/overcloud_node.py
index e354f8e..13d60af 100644
--- a/v1/overcloud_node.py
+++ b/v1/overcloud_node.py
@@ -221,8 +221,6 @@ class ImportNode(command.Command):
             self.app.client_manager,
             nodes_json=nodes_config,
             queue_name=queue_name,
-            kernel_name=deploy_kernel,
-            ramdisk_name=deploy_ramdisk,
             instance_boot_option=parsed_args.instance_boot_option
         )

-- 
1.8.3.1
__EOF__

[stack@oscloud5 ~]$ (cd /usr/lib/python2.7/site-packages/tripleo_common/; cat << '__EOF__' | sudo git am)
From 9b8312d8940fdda85e5049629f860d9a4c19110d Mon Sep 17 00:00:00 2001
From: root <root@oscloud5.stglabs.ibm.com>
Date: Wed, 1 Nov 2017 16:56:35 +0000
Subject: [PATCH] tripleo_common

---
 actions/baremetal.py   |  9 ++-------
 actions/validations.py | 45 ++++++++++++++++++++++++++++++++++++++-------
 utils/nodes.py         | 37 +++++++++++++++++++++++++++----------
 3 files changed, 67 insertions(+), 24 deletions(-)

diff --git a/actions/baremetal.py b/actions/baremetal.py
index dc73355..979752c 100644
--- a/actions/baremetal.py
+++ b/actions/baremetal.py
@@ -43,14 +43,11 @@ class RegisterOrUpdateNodes(base.TripleOAction):
     :return: list of node objects representing the new nodes.
     """

-    def __init__(self, nodes_json, remove=False, kernel_name=None,
-                 ramdisk_name=None, instance_boot_option='local'):
+    def __init__(self, nodes_json, remove=False, instance_boot_option='local'):
         super(RegisterOrUpdateNodes, self).__init__()
         self.nodes_json = nodes_json
         self.remove = remove
         self.instance_boot_option = instance_boot_option
-        self.kernel_name = kernel_name
-        self.ramdisk_name = ramdisk_name

     def run(self, context):
         for node in self.nodes_json:
@@ -67,9 +64,7 @@ class RegisterOrUpdateNodes(base.TripleOAction):
                 self.nodes_json,
                 client=baremetal_client,
                 remove=self.remove,
-                glance_client=image_client,
-                kernel_name=self.kernel_name,
-                ramdisk_name=self.ramdisk_name)
+                glance_client=image_client)
         except Exception as err:
             LOG.exception("Error registering nodes with ironic.")
             return actions.Result(error=six.text_type(err))
diff --git a/actions/validations.py b/actions/validations.py
index 017f99d..28ae86c 100644
--- a/actions/validations.py
+++ b/actions/validations.py
@@ -265,31 +265,62 @@ class CheckNodeBootConfigurationAction(base.TripleOAction):

     # TODO(bcrochet): The validation actions are temporary. This logic should
     #                 move to the tripleo-validations project eventually.
-    def __init__(self, node, kernel_id, ramdisk_id):
+    def __init__(self, node, images):
         super(CheckNodeBootConfigurationAction, self).__init__()

         self.node = node
-        self.kernel_id = kernel_id
-        self.ramdisk_id = ramdisk_id
+        self.images = images
+
+    def _check_for_image(self, name):
+        found_images = [item['id'] for item in self.images
+                        if item['name'] == name]
+        if len(found_images) == 1:
+            return found_images[0]
+        else:
+            return None

     def run(self, context):
         warnings = []
         errors = []
+
+        cpu_arch = None
+        if self.node is None:
+            errors.append("self.node is None ")
+        elif 'properties' in self.node:
+            if 'cpu_arch' in self.node['properties']:
+                cpu_arch = self.node['properties']['cpu_arch']
+            else:
+                errors.append("cpu_arch not in properties (%s) " % self.node['properties'])
+        else:
+            errors.append("properties not in node (%s)" % self.node)
+
+        kernel_name = 'bm-deploy-kernel'
+        ramdisk_name = 'bm-deploy-ramdisk'
+        if cpu_arch is not None:
+            kernel_name = cpu_arch+'-bm-deploy-kernel'
+            ramdisk_name = cpu_arch+'-bm-deploy-ramdisk'
+        kernel_id = self._check_for_image(kernel_name)
+        ramdisk_id = self._check_for_image(ramdisk_name)
+        if kernel_id is None:
+            errors.append("kernel_id = %s, kernel_name = %s" % (kernel_id,kernel_name,))
+        if ramdisk_id is None:
+            errors.append("ramdisk_id = %s, ramdisk_name= %s" % (ramdisk_id,ramdisk_name,))
+
         message = ("Node {uuid} has an incorrectly configured "
                    "{property}. Expected \"{expected}\" but got "
                    "\"{actual}\".")
-        if self.node['driver_info'].get('deploy_ramdisk') != self.ramdisk_id:
+        if self.node['driver_info'].get('deploy_ramdisk') != ramdisk_id:
             errors.append(message.format(
                 uuid=self.node['uuid'],
                 property='driver_info/deploy_ramdisk',
-                expected=self.ramdisk_id,
+                expected=ramdisk_id,
                 actual=self.node['driver_info'].get('deploy_ramdisk')
             ))
-        if self.node['driver_info'].get('deploy_kernel') != self.kernel_id:
+        if self.node['driver_info'].get('deploy_kernel') != kernel_id:
             errors.append(message.format(
                 uuid=self.node['uuid'],
                 property='driver_info/deploy_kernel',
-                expected=self.kernel_id,
+                expected=kernel_id,
                 actual=self.node['driver_info'].get('deploy_kernel')
             ))
         capabilities = nodeutils.capabilities_to_dict(
diff --git a/utils/nodes.py b/utils/nodes.py
index 0d93804..edad508 100644
--- a/utils/nodes.py
+++ b/utils/nodes.py
@@ -23,6 +23,9 @@ from oslo_concurrency import processutils
 from tripleo_common import exception
 from tripleo_common.utils import glance

+from glanceclient.v2.client import Client as real_glance_client
+from glanceclient import exc as exceptions
+
 LOG = logging.getLogger(__name__)

 _KNOWN_INTERFACE_FIELDS = [
@@ -449,8 +452,7 @@ def _clean_up_extra_nodes(seen, client, remove=False):
             LOG.debug('Extra registered node %s found.' % node)


-def register_all_nodes(nodes_list, client, remove=False, glance_client=None,
-                       kernel_name=None, ramdisk_name=None):
+def register_all_nodes(nodes_list, client, remove=False, glance_client=None):
     """Register all nodes in nodes_list in the baremetal service.

     :param nodes_list: The list of nodes to register.
@@ -465,17 +467,32 @@ def register_all_nodes(nodes_list, client, remove=False, glance_client=None,
     LOG.debug('Registering all nodes.')
     node_map = _populate_node_mapping(client)

-    glance_ids = {'kernel': None, 'ramdisk': None}
-    if kernel_name and ramdisk_name:
-        glance_ids = glance.create_or_find_kernel_and_ramdisk(
-            glance_client, kernel_name, ramdisk_name)
+    def find_image(name, format):
+        if not isinstance(glance_client, real_glance_client):
+            raise exceptions.NotFound("MARK1")
+
+        images = glance_client.images.list(name=name,
+                                           disk_format=format)
+        for img in images:
+            if img['name'] == name and img['disk_format'] == format:
+                return img['id']
+        return None

     seen = []
     for node in nodes_list:
-        if glance_ids['kernel'] and 'kernel_id' not in node:
-            node['kernel_id'] = glance_ids['kernel']
-        if glance_ids['ramdisk'] and 'ramdisk_id' not in node:
+
+        kernel_id = find_image(name=node['arch']+'-bm-deploy-kernel',
+                               format='aki')
+        if not kernel_id:
+            raise exceptions.NotFound("No image found for "+kernel_name)
+
+        ramdisk_id = find_image(name=node['arch']+'-bm-deploy-ramdisk',
+                                format='ari')
+        if not ramdisk_id:
+            raise exceptions.NotFound("No image found for "+ramdisk_name)
+
+        node['kernel_id'] = kernel_id
+        node['ramdisk_id'] = ramdisk_id

         node = _update_or_register_ironic_node(node, node_map, client=client)
         seen.append(node)
-- 
1.8.3.1
__EOF__

[stack@undercloud ~]$ (for SERVICE in openstack-mistral-api.service openstack-mistral-engine.service openstack-mistral-executor.service; do sudo systemctl restart ${SERVICE}; done)

Ironic needs some different settings to be able to support PXE for ppc64le:

[stack@oscloud5 ~]$ (cd /etc/ironic; cat << '__EOF__' | sudo patch -p0)
--- ironic.conf.orig    2017-11-03 13:54:06.440644679 +0000
+++ ironic.conf 2017-11-03 15:01:47.584431653 +0000
@@ -343,6 +343,7 @@
 # for this option to be unset. (string value)
 # Allowed values: debug, info, warning, error, critical
 #notification_level = <None>
+notification_level = debug
 
 # Directory where the ironic python module is installed.
 # (string value)
@@ -3506,7 +3507,7 @@
 # On ironic-conductor node, template file for PXE
 # configuration. (string value)
 #pxe_config_template = $pybasedir/drivers/modules/pxe_config.template
-pxe_config_template=$pybasedir/drivers/modules/ipxe_config.template
+pxe_config_template=$pybasedir/drivers/modules/pxe_config.template
 
 # On ironic-conductor node, template file for PXE
 # configuration for UEFI boot loader. (string value)
@@ -3517,6 +3518,7 @@
 # configuration per node architecture. For example:
 # aarch64:/opt/share/grubaa64_pxe_config.template (dict value)
 #pxe_config_template_by_arch =
+pxe_config_template_by_arch = ppc64le:$pybasedir/drivers/modules/pxe_config.template
 
 # IP address of ironic-conductor node's TFTP server. (string
 # value)
@@ -3547,7 +3549,7 @@
 
 # Bootfile DHCP parameter. (string value)
 #pxe_bootfile_name = pxelinux.0
-pxe_bootfile_name=undionly.kpxe
+pxe_bootfile_name=pxelinux.0
 
 # Bootfile DHCP parameter for UEFI boot mode. (string value)
 #uefi_pxe_bootfile_name = bootx64.efi
@@ -3556,10 +3558,11 @@
 # Bootfile DHCP parameter per node architecture. For example:
 # aarch64:grubaa64.efi (dict value)
 #pxe_bootfile_name_by_arch =
+pxe_bootfile_name_by_arch = ppc64le:config
 
 # Enable iPXE boot. (boolean value)
 #ipxe_enabled = false
-ipxe_enabled=True
+ipxe_enabled = false
 
 # On ironic-conductor node, the path to the main iPXE script
 # file. (string value)
__EOF__
[stack@oscloud5 ~]$ for I in openstack-ironic-conductor.service openstack-ironic-inspector.service openstack-ironic-inspector-dnsmasq.service; do sudo systemctl restart ${I}; done

I then go through the process of installing the overcloud:

[stack@oscloud5 ~]$ source stackrc
(undercloud) [stack@oscloud5 ~]$ time openstack overcloud image upload
...

The overcloud-full qcow2 image needs to be recreated in glance so that it loses both the kernel_id and the ramdisk_id. This way a full disk image can be deployed.

(undercloud) [stack@oscloud5 ~]$ (FILE="overcloud-full.qcow2"; UUID=$(openstack image list -f value | grep 'overcloud-full ' | awk '{print $1;}'); openstack image delete ${UUID}; openstack image create --container-format bare --disk-format qcow2 --min-disk 0 --min-ram 0 --file ${FILE} --public overcloud-full)

Now import the baremetal nodes and assign them profiles.

(undercloud) [stack@oscloud5 ~]$ time openstack overcloud node import --provide instackenv.json 2>&1 | tee output.overcloud.node.import
...
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
| Node UUID                            | Node Name | Provision State | Current Profile | Possible Profiles |
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
| ff2fdac5-6cc5-47a9-a095-d942b3960795 |           | available       | None            |                   |
| ef3d7b3b-97b8-42ab-b501-896474df658f |           | available       | None            |                   |
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
(undercloud) [stack@oscloud5 ~]$ (COMPUTE=""; CONTROL=""; while IFS=$' ' read -r -a PROFILES; do NODE=${PROFILES[0]}; ARCH=$(openstack baremetal node show --fields properties --format shell ${NODE} | sed -r -e "s,^.*cpu_arch': u'([^']*).*$,\1,"); echo "ARCH=${ARCH} NODE=${NODE}"; if [ -z "${COMPUTE}" -a "${ARCH}" == "ppc64le" ]; then COMPUTE=${NODE}; echo "Compute:"; ironic node-update ${COMPUTE} replace properties/capabilities=profile:compute,boot_option:local; continue; elif [ -z "${CONTROL}" -a "${ARCH}" == "x86_64" ]; then CONTROL=${PROFILES[0]}; echo "Control:"; ironic node-update ${CONTROL} replace properties/capabilities=profile:control,boot_option:local; continue; else echo "Error: ${NODE}"; openstack baremetal node show --fields properties --format shell ${NODE}; fi; done < <(openstack overcloud profiles list -f value))
(undercloud) [stack@oscloud5 ~]$ openstack overcloud profiles list
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
| Node UUID                            | Node Name | Provision State | Current Profile | Possible Profiles |
+--------------------------------------+-----------+-----------------+-----------------+-------------------+
| ff2fdac5-6cc5-47a9-a095-d942b3960795 |           | available       | compute         |                   |
| ef3d7b3b-97b8-42ab-b501-896474df658f |           | available       | control         |                   |
+--------------------------------------+-----------+-----------------+-----------------+-------------------+

Patch the openstack-tripleo-heat-templates locally with our configuration and then do the deploy.

(undercloud) [stack@oscloud5 ~]$ cp -r /usr/share/openstack-tripleo-heat-templates templates
(undercloud) [stack@oscloud5 ~]$ (cd templates/; curl --silent -o - https://hamzy.fedorapeople.org/openstack-tripleo-heat-templates.patch | patch -p1)
(undercloud) [stack@oscloud5 ~]$ time openstack overcloud deploy --debug --templates /home/stack/templates -e /home/stack/templates/environments/network-environment.yaml -e /home/stack/templates/environments/network-isolation-custom.yaml --control-scale 1 --compute-scale 1 --control-flavor control --compute-flavor compute 2>&1 | tee output.overcloud.deploy