No edit summary |
No edit summary |
||
(8 intermediate revisions by the same user not shown) | |||
Line 125: | Line 125: | ||
[hamzy@pkvmci853 mockchain.pike]$ rm mozjs38-38.8.0-2.el7.src.rpm | [hamzy@pkvmci853 mockchain.pike]$ rm mozjs38-38.8.0-2.el7.src.rpm | ||
</pre> | </pre> | ||
=== etcd === | |||
etcd has an exclusive arch that can be removed and rebuilt easily. | |||
<pre> | |||
[hamzy@pkvmci853 ~]$ (TMP1=$(mktemp); TMP2=$(mktemp); rpm2cpio ~/mockchain.pike/etcd-2.0.9-1.mh.el7.centos.src.rpm | cpio --extract --to-stdout etcd.spec > ${TMP2} 2>/dev/null; wget --quiet -O - 'http://cbs.centos.org/kojifiles/packages/etcd/2.0.9/1.el7/src/etcd-2.0.9-1.el7.src.rpm' | rpm2cpio | cpio --extract --to-stdout etcd.spec > ${TMP1} 2>/dev/null; diff -Nu ${TMP1} ${TMP2}; /bin/rm ${TMP1} ${TMP2}) | |||
--- /tmp/tmp.nlnPmsXWoR 2017-07-13 08:29:09.986373084 -0400 | |||
+++ /tmp/tmp.EzsacdS6g0 2017-07-13 08:29:09.856371472 -0400 | |||
@@ -14,7 +14,7 @@ | |||
Name: %{repo} | |||
Version: 2.0.9 | |||
-Release: 1%{?dist} | |||
+Release: 1.mh%{?dist} | |||
Summary: A highly-available key value store for shared configuration | |||
License: ASL 2.0 | |||
URL: https://%{import_path} | |||
@@ -26,7 +26,7 @@ | |||
Patch0: etcd-2.0.1-Replace-depricated-ErrWrongType-with-its-local-defin.patch | |||
%endif | |||
-ExclusiveArch: %{ix86} x86_64 %{arm} | |||
+ExclusiveArch: %{ix86} x86_64 %{arm} ppc64le | |||
BuildRequires: golang >= 1.2.1-3 | |||
%if 0%{?fedora} | |||
BuildRequires: golang(code.google.com/p/gogoprotobuf/proto) | |||
[hamzy@pkvmci853 ~]$ cbs build --scratch cloud7-openstack-common-el7 ~/mockchain.pike/etcd-2.0.9-1.mh.el7.centos.src.rpm | |||
... | |||
Created task: 190223 | |||
Task info: https://cbs.centos.org/koji/taskinfo?taskID=190223 | |||
... | |||
190223 build (cloud7-openstack-common-el7, etcd-2.0.9-1.mh.el7.centos.src.rpm) completed successfully | |||
</pre> | |||
=== libunwind === | |||
libunwind-1.1-3.el7.src.rpm has an exclusive arch but libunwind-1.1-10.el7.src.rpm does not | |||
<pre> | |||
[hamzy@pkvmci853 ~]$ cbs build --scratch cloud7-openstack-common-el7 ~/mockchain.pike/libunwind-1.1-10.el7.src.rpm | |||
... | |||
Created task: 190226 | |||
Task info: https://cbs.centos.org/koji/taskinfo?taskID=190226 | |||
... | |||
190226 build (cloud7-openstack-common-el7, libunwind-1.1-10.el7.src.rpm) completed successfully | |||
</pre> | |||
=== mozjs === | |||
There seems to be two mozjs rpms. Delete mozjs38-38.8.0-2.el7.src.rpm. | |||
=== nodejs === | === nodejs === | ||
Line 136: | Line 187: | ||
... | ... | ||
</pre> | </pre> | ||
=== openstack-tripleo-ui-deps === | |||
Documented more here [https://fedoraproject.org/wiki/User:Hamzy/TripleO_ppc64le_issues#Packages:_openstack-tripleo-ui-deps] | |||
=== python-cradox === | |||
python-cradox-1.3.2-1.el7.src.rpm has an exclusive arch. However, python-cradox-1.3.2-1.fc27.src.rpm does not. | |||
<pre> | |||
[hamzy@pkvmci853 ~]$ (TMP1=$(mktemp); TMP2=$(mktemp); wget --quiet -O - 'https://kojipkgs.fedoraproject.org//packages/python-cradox/1.3.2/1.fc27/src/python-cradox-1.3.2-1.fc27.src.rpm' | rpm2cpio | cpio --extract --to-stdout python-cradox.spec > ${TMP2} 2>/dev/null; wget --quiet -O - 'http://cbs.centos.org/kojifiles/packages/python-cradox/1.3.2/1.el7/src/python-cradox-1.3.2-1.el7.src.rpm' | rpm2cpio | cpio --extract --to-stdout python-cradox.spec > ${TMP1} 2>/dev/null; diff -Nu ${TMP1} ${TMP2}; /bin/rm ${TMP1} ${TMP2}) | |||
--- /tmp/tmp.0ZRAen8Syl 2017-07-13 08:56:47.126859218 -0400 | |||
+++ /tmp/tmp.WwM9dpl8gL 2017-07-13 08:56:47.036858102 -0400 | |||
@@ -9,9 +9,6 @@ | |||
Release: 1%{?dist} | |||
Summary: Python libraries for the Ceph librados library with use cython instead of ctypes | |||
-# TODO(apevec) temp disable ppc64le, builder was not picking up the task after 1h | |||
-ExclusiveArch: x86_64 aarch64 | |||
- | |||
License: LGPLv2 | |||
URL: https://github.com/sileht/pycradox | |||
Source0: https://files.pythonhosted.org/packages/source/c/%{pypi_name}/%{pypi_name}-%{version}.tar.gz | |||
</pre> | |||
However, a build of python-cradox-1.3.2-1.fc27.src.rpm results in: | |||
<pre> | |||
... | |||
DEBUG util.py:417: Error: No Package found for librados2-devel | |||
... | |||
</pre> | |||
=== python-falcon === | |||
python-falcon-1.0.0-4.el7.src.rpm will hang the mockchain/mock process during the build. | |||
=== python-httplib2 === | === python-httplib2 === | ||
python-httplib2-0.9.2-1.el7.src.rpm has a hardcoded python3 dependency in the spec file when CentOS uses python34 | python-httplib2-0.9.2-1.el7.src.rpm has a hardcoded python3 dependency in the spec file when CentOS uses python34. python-httplib2-0.9.1-2.el7.src.rpm has ifdefed python3 to not build. | ||
<pre> | <pre> | ||
Line 146: | Line 233: | ||
BuildRequires: python-devel | BuildRequires: python-devel | ||
BuildRequires: python3-devel | BuildRequires: python3-devel | ||
</pre> | |||
Apparently CentOS does not use EPEL when building packages, so you can't use epel-7-ppc64le.cfg and instead a custom mock environment needs to be created. | |||
=== python-mock === | |||
Has hardcoded python3 requires. However, CentOS uses python34. | |||
<pre> | |||
DEBUG util.py:439: Error: No Package found for python3-funcsigs | |||
DEBUG util.py:439: Error: No Package found for python3-pbr | |||
DEBUG util.py:439: Error: No Package found for python34-unittest2 | |||
</pre> | |||
Apparently CentOS does not use EPEL when building packages, so you can't use epel-7-ppc64le.cfg and instead a custom mock environment needs to be created. | |||
=== python-oslo-config and python-stevedore === | |||
OpenStack has some pretty strict dependencies that are not coded into the RPM spec files. Building with a newer version of python-stevedore seems to fix this issue: | |||
<pre> | |||
[hamzy@pkvmci853 ~]$ cat ~/mockchain.pike/mock-repo/results/epel-7-ppc64le/python-oslo-config-4.1.1-0.20170517115009.3290830.el7.centos/build.log | |||
... | |||
Exception occurred: | |||
File "/builddir/build/BUILD/oslo.config-4.1.1/oslo_config/generator.py", line 407, in _get_opt_default_updaters | |||
invoke_on_load=False) | |||
TypeError: __init__() got an unexpected keyword argument 'warn_on_missing_entrypoint' | |||
... | |||
</pre> | |||
=== python-sphinxcontrib-programoutput and python-sphinxcontrib-websupport === | |||
Both of these rpms have circular dependencies! For example: | |||
<pre> | |||
... | |||
DEBUG util.py:439: ---> Package python2-sphinx.noarch 0:1.6.2-3.el7.centos will be installed | |||
DEBUG util.py:439: --> Processing Dependency: python2-sphinxcontrib-websupport for package: python2-sphinx-1.6.2-3.el7.centos.noarch | |||
DEBUG util.py:439: --> Finished Dependency Resolution | |||
DEBUG util.py:439: Error: Package: python2-sphinx-1.6.2-3.el7.centos.noarch (local_build_repo) | |||
DEBUG util.py:439: Requires: python2-sphinxcontrib-websupport | |||
DEBUG util.py:439: You could try using --skip-broken to work around the problem | |||
DEBUG util.py:439: You could try running: rpm -Va --nofiles --nodigest | |||
DEBUG util.py:577: Child return code was: 1 | |||
... | |||
</pre> | </pre> |
Latest revision as of 12:59, 13 July 2017
goal
The goal is to build the following repository: https://buildlogs.centos.org/centos/7/cloud/ppc64le/openstack-pike/
Downloading the packages
Unfortunately, there is no corresponding SRPM directory to directly download the source packages. So I think I need to download all packages tagged with cloud7-openstack-pike-el7-build from CBS.
while read SRPM do echo "Downloading ${SRPM}" ((TRIES=5)) while ((TRIES > 0)) do cbs download-build --arch=src ${SRPM} if [ $? -eq 0 ] then break fi ((TRIES -= 1)) done; done < <(cbs list-tagged cloud7-openstack-pike-el7-build --rpms --arch=src --inherit --latest-n=1)
Building the packages
The simplest way (although not the fastest) is to build them all with mockchain.
mockchain -r epel-7-ppc64le --tmp_prefix=mockbuild --localrepo=mock-repo --log=mockchain.log --recurse *.src.rpm
Unfortunately, they all do not build:
[hamzy@pkvmci853 ~]$ (SUCCESS=$(find ~/mockchain.pike/ -name success | wc -l); FAIL=$(find ~/mockchain.pike/ -name fail | wc -l); echo "${SUCCESS} packages built successfully and ${FAIL} packages failed") 473 packages built successfully and 71 packages failed
Some of the root packages needed are:
[hamzy@pkvmci853 ~]$ while read FAIL; do DIR=$(dirname ${FAIL}); grep --ignore-case --no-filename 'No Package found' ${DIR}/root.log | sed 's,^.*No Package found for ,,' | sort | uniq; done < <(find ~/mockchain.pike/ -name fail) nodejs etcd python3-devel python34-unittest2 python3-funcsigs python3-pbr python-oslo-i18n python-oslotest python-oslo-config python-oslotest
Pulling in other, already built repositories, and building again:
[hamzy@pkvmci853 ~]$ rm -rf ~/mockchain.pike/mock-repo/results/epel-7-ppc64le/ [hamzy@pkvmci853 ~]$ (export ADDREPO="--addrepo=https://cbs.centos.org/repos/cloud7-openstack-common-release/ppc64le/os/ --addrepo=https://cbs.centos.org/repos/cloud7-openstack-common-candidate/ppc64le/os/"; cd mockchain.pike/; time mockchain -r fast-epel-7-ppc64le --tmp_prefix=mockbuild --localrepo=mock-repo ${ADDREPO} --log=mockchain.log --recurse *.src.rpm) real 1135m20.712s user 1760m16.636s sys 163m49.094s [hamzy@pkvmci853 ~]$ (SUCCESS=$(find ~/mockchain.pike/ -name success | wc -l); FAIL=$(find ~/mockchain.pike/ -name fail | wc -l); echo "${SUCCESS} packages built successfully and ${FAIL} packages failed") 510 packages built successfully and 31 packages failed
is a little better. The following packages fail to build:
[hamzy@pkvmci853 ~]$ while read FAIL; do A=$(dirname ${FAIL}); B=$(basename ${A}); echo ${B}; done < <(find ~/mockchain.pike/ -name fail) mozjs38-38.8.0-2.el7 mariadb-galera-5.5.40-3.el7 nodejs-6.10.2-1.el7 libunwind-1.1-3.el7 dibbler-1.0.1-0.RC1.2.el7 erlang-rebar-2.1.0-0.0.el7 mongodb-3.2.8-3.el7 gdal-1.11.4-1.el7 openstack-tripleo-ui-deps-7-5.el7 phantomjs-1.9.7-3.el7 python-cassandra-driver-3.7.1-4.el7 python-cradox-1.3.2-1.el7 python-etcd-0.4.3-5.el7 python-flake8-2.4.1-2.el7 python-flask-babel-0.9-1.el7 python-httplib2-0.9.2-1.el7 python-influxdb-4.0.0-1.el7.1 python-kubernetes-2.0.0-1.el7 python-mock-2.0.0-1.el7 python-murano-pkg-check-0.3.0-2.el7 python-nose-1.3.7-7.el7 python-openstackdocstheme-1.11.0-1.el7 python-pyasn1-0.1.9-6.el7.1 python-ryu-4.13-2.el7 python-sphinxcontrib-programoutput-0.8-6.el7 python-sphinxcontrib-websupport-1.0.1-1.el7 python-statsd-3.2.1-5.el7 python-tables-3.1.1-2.el7 python-tinyrpc-0.5-4.20170523git1f38ac.el7 python-tosca-parser-0.7.0-1.el7 python-urllib3-1.16-1.el7 qhull-2015.2-1.el7 qpid-dispatch-0.6.1-4.el7 subunit-1.0.0-1.el7 v8-3.14.5.10-14.el7
Work arounds
There are some work arounds you can perform:
[hamzy@pkvmci853 mockchain.pike]$ rm nodejs-6.10.2-1.el7.src.rpm; cp ~/mockchain2/nodejs-6.10.2-1.mh.el7.centos.src.rpm . [hamzy@pkvmci853 mockchain.pike]$ rm python-falcon-1.0.0-4.el7.src.rpm [hamzy@pkvmci853 mockchain.pike]$ rm python-httplib2-0.9.2-1.el7.src.rpm; wget http://cbs.centos.org/kojifiles/packages/python-httplib2/0.9.1/2.el7/src/python-httplib2-0.9.1-2.el7.src.rpm [hamzy@pkvmci853 mockchain.pike]$ rm etcd-2.0.9-1.el7.src.rpm; cp ~/mockchain2/etcd-2.0.9-1.mh.el7.centos.src.rpm . [hamzy@pkvmci853 mockchain.pike]$ rm libunwind-1.1-3.el7.src.rpm; wget https://kojipkgs.fedoraproject.org//packages/libunwind/1.1/10.el7/src/libunwind-1.1-10.el7.src.rpm; [hamzy@pkvmci853 mockchain.pike]$ rm python-stevedore-1.8.0-1.el7.src.rpm; wget http://cbs.centos.org/kojifiles/packages/python-stevedore/1.20.0/1.el7/src/python-stevedore-1.20.0-1.el7.src.rpm [hamzy@pkvmci853 mockchain.pike]$ rm openstack-tripleo-ui-deps-7-5.el7.src.rpm; cp ~/mockchain2/openstack-tripleo-ui-deps-7-5.mh.el7.centos.src.rpm . [hamzy@pkvmci853 mockchain.pike]$ rm python-cradox-1.3.2-1.el7.src.rpm; wget https://kojipkgs.fedoraproject.org//packages/python-cradox/1.3.2/1.fc27/src/python-cradox-1.3.2-1.fc27.src.rpm [hamzy@pkvmci853 mockchain.pike]$ rm mozjs38-38.8.0-2.el7.src.rpm
etcd
etcd has an exclusive arch that can be removed and rebuilt easily.
[hamzy@pkvmci853 ~]$ (TMP1=$(mktemp); TMP2=$(mktemp); rpm2cpio ~/mockchain.pike/etcd-2.0.9-1.mh.el7.centos.src.rpm | cpio --extract --to-stdout etcd.spec > ${TMP2} 2>/dev/null; wget --quiet -O - 'http://cbs.centos.org/kojifiles/packages/etcd/2.0.9/1.el7/src/etcd-2.0.9-1.el7.src.rpm' | rpm2cpio | cpio --extract --to-stdout etcd.spec > ${TMP1} 2>/dev/null; diff -Nu ${TMP1} ${TMP2}; /bin/rm ${TMP1} ${TMP2}) --- /tmp/tmp.nlnPmsXWoR 2017-07-13 08:29:09.986373084 -0400 +++ /tmp/tmp.EzsacdS6g0 2017-07-13 08:29:09.856371472 -0400 @@ -14,7 +14,7 @@ Name: %{repo} Version: 2.0.9 -Release: 1%{?dist} +Release: 1.mh%{?dist} Summary: A highly-available key value store for shared configuration License: ASL 2.0 URL: https://%{import_path} @@ -26,7 +26,7 @@ Patch0: etcd-2.0.1-Replace-depricated-ErrWrongType-with-its-local-defin.patch %endif -ExclusiveArch: %{ix86} x86_64 %{arm} +ExclusiveArch: %{ix86} x86_64 %{arm} ppc64le BuildRequires: golang >= 1.2.1-3 %if 0%{?fedora} BuildRequires: golang(code.google.com/p/gogoprotobuf/proto) [hamzy@pkvmci853 ~]$ cbs build --scratch cloud7-openstack-common-el7 ~/mockchain.pike/etcd-2.0.9-1.mh.el7.centos.src.rpm ... Created task: 190223 Task info: https://cbs.centos.org/koji/taskinfo?taskID=190223 ... 190223 build (cloud7-openstack-common-el7, etcd-2.0.9-1.mh.el7.centos.src.rpm) completed successfully
libunwind
libunwind-1.1-3.el7.src.rpm has an exclusive arch but libunwind-1.1-10.el7.src.rpm does not
[hamzy@pkvmci853 ~]$ cbs build --scratch cloud7-openstack-common-el7 ~/mockchain.pike/libunwind-1.1-10.el7.src.rpm ... Created task: 190226 Task info: https://cbs.centos.org/koji/taskinfo?taskID=190226 ... 190226 build (cloud7-openstack-common-el7, libunwind-1.1-10.el7.src.rpm) completed successfully
mozjs
There seems to be two mozjs rpms. Delete mozjs38-38.8.0-2.el7.src.rpm.
nodejs
nodejs has an exclusive arch that can be removed and rebuilt easily.
[hamzy@pkvmci853 mockchain.pike]$ mockchain -r epel-7-ppc64le --tmp_prefix=mockbuild --localrepo=mock-repo ${ADDREPO} --log=mockchain.log --recurse nodejs-6.10.2-1.el7.src.rpm ... error: Architecture is not included: ppc64le ...
openstack-tripleo-ui-deps
Documented more here [1]
python-cradox
python-cradox-1.3.2-1.el7.src.rpm has an exclusive arch. However, python-cradox-1.3.2-1.fc27.src.rpm does not.
[hamzy@pkvmci853 ~]$ (TMP1=$(mktemp); TMP2=$(mktemp); wget --quiet -O - 'https://kojipkgs.fedoraproject.org//packages/python-cradox/1.3.2/1.fc27/src/python-cradox-1.3.2-1.fc27.src.rpm' | rpm2cpio | cpio --extract --to-stdout python-cradox.spec > ${TMP2} 2>/dev/null; wget --quiet -O - 'http://cbs.centos.org/kojifiles/packages/python-cradox/1.3.2/1.el7/src/python-cradox-1.3.2-1.el7.src.rpm' | rpm2cpio | cpio --extract --to-stdout python-cradox.spec > ${TMP1} 2>/dev/null; diff -Nu ${TMP1} ${TMP2}; /bin/rm ${TMP1} ${TMP2}) --- /tmp/tmp.0ZRAen8Syl 2017-07-13 08:56:47.126859218 -0400 +++ /tmp/tmp.WwM9dpl8gL 2017-07-13 08:56:47.036858102 -0400 @@ -9,9 +9,6 @@ Release: 1%{?dist} Summary: Python libraries for the Ceph librados library with use cython instead of ctypes -# TODO(apevec) temp disable ppc64le, builder was not picking up the task after 1h -ExclusiveArch: x86_64 aarch64 - License: LGPLv2 URL: https://github.com/sileht/pycradox Source0: https://files.pythonhosted.org/packages/source/c/%{pypi_name}/%{pypi_name}-%{version}.tar.gz
However, a build of python-cradox-1.3.2-1.fc27.src.rpm results in:
... DEBUG util.py:417: Error: No Package found for librados2-devel ...
python-falcon
python-falcon-1.0.0-4.el7.src.rpm will hang the mockchain/mock process during the build.
python-httplib2
python-httplib2-0.9.2-1.el7.src.rpm has a hardcoded python3 dependency in the spec file when CentOS uses python34. python-httplib2-0.9.1-2.el7.src.rpm has ifdefed python3 to not build.
[hamzy@pkvmci853 ~]$ wget --quiet -O - 'http://cbs.centos.org/kojifiles/packages/python-httplib2/0.9.2/1.el7/src/python-httplib2-0.9.2-1.el7.src.rpm' | rpm2cpio | cpio --extract --to-stdout python-httplib2.spec 2>/dev/null | grep 'BuildRequires:' BuildRequires: python-setuptools BuildRequires: python-devel BuildRequires: python3-devel
Apparently CentOS does not use EPEL when building packages, so you can't use epel-7-ppc64le.cfg and instead a custom mock environment needs to be created.
python-mock
Has hardcoded python3 requires. However, CentOS uses python34.
DEBUG util.py:439: Error: No Package found for python3-funcsigs DEBUG util.py:439: Error: No Package found for python3-pbr DEBUG util.py:439: Error: No Package found for python34-unittest2
Apparently CentOS does not use EPEL when building packages, so you can't use epel-7-ppc64le.cfg and instead a custom mock environment needs to be created.
python-oslo-config and python-stevedore
OpenStack has some pretty strict dependencies that are not coded into the RPM spec files. Building with a newer version of python-stevedore seems to fix this issue:
[hamzy@pkvmci853 ~]$ cat ~/mockchain.pike/mock-repo/results/epel-7-ppc64le/python-oslo-config-4.1.1-0.20170517115009.3290830.el7.centos/build.log ... Exception occurred: File "/builddir/build/BUILD/oslo.config-4.1.1/oslo_config/generator.py", line 407, in _get_opt_default_updaters invoke_on_load=False) TypeError: __init__() got an unexpected keyword argument 'warn_on_missing_entrypoint' ...
python-sphinxcontrib-programoutput and python-sphinxcontrib-websupport
Both of these rpms have circular dependencies! For example:
... DEBUG util.py:439: ---> Package python2-sphinx.noarch 0:1.6.2-3.el7.centos will be installed DEBUG util.py:439: --> Processing Dependency: python2-sphinxcontrib-websupport for package: python2-sphinx-1.6.2-3.el7.centos.noarch DEBUG util.py:439: --> Finished Dependency Resolution DEBUG util.py:439: Error: Package: python2-sphinx-1.6.2-3.el7.centos.noarch (local_build_repo) DEBUG util.py:439: Requires: python2-sphinxcontrib-websupport DEBUG util.py:439: You could try using --skip-broken to work around the problem DEBUG util.py:439: You could try running: rpm -Va --nofiles --nodigest DEBUG util.py:577: Child return code was: 1 ...