From Fedora Project Wiki
No edit summary
Line 26: Line 26:
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. -->
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. -->


A revisit of the [[Changes/System Python|System Python]] change from [[Releases/24 | Fedora 24]]. It has been renamed from '''System Python''' to '''Platform Python''' not to collide with upstream [https://www.python.org/dev/peps/pep-0432/#a-system-python-executable PEP 432].
A revisit of the [[Changes/System Python|System Python]] change from [[Releases/24 | Fedora 24]]. Platform Python will be a separate stack fo Python packages aimed to provide all necessary dependencies to run and build dnf. In [[Releases/27 | Fedora 27]] this will be Python 3.6 (same version as {{package|python3}}, but this might change in the future with the Modularity effort).
 
Platform Python will be a separate stack fo Python packages living in the Platform module aimed to provide all necessary dependencies to run and build dnf. In [[Releases/27 | Fedora 27]] this will be Python 3.6 (same version as {{package|python3}}, but this might change in the future with the Modularity effort).


== Owner ==
== Owner ==
Line 66: Line 64:
The changes made during the [[Changes/System Python|System Python]] change from [[Releases/24 | Fedora 24]] will be reverted.
The changes made during the [[Changes/System Python|System Python]] change from [[Releases/24 | Fedora 24]] will be reverted.


A new platform-python (source) package will be created as a copy of the {{package|python3}} package. It will only be built as part of the Platform module. Unlike python3, it will be changed so that:
A new platform-python (source) package will be created as a copy of python3 package. Unlike python3, it will be changed so that:


* the location of files will not be <code>/usr/lib/python3.6/</code> but <code>/usr/lib/platform-python/</code>
* the location of files will not be <code>/usr/lib/python3.6/</code> but <code>/usr/lib/platform-python/</code>
** the script that generates the <code>python(abi)</code> requirement will be adapted to generate <code>platform-python(abi)</code> requirement for that location
** the script that generates the <code>python(abi)</code> requirement will be adapted to generate <code>platform-python(abi)</code> requirement for that location
* the location of the binary will be <code>/usr/libexec/platform-python</code>
* the location of the binary will be <code>/usr/libexec/platform-python</code>
* all the macros will use the platform python binary and will be renamed
* all subpackages will be named <code>platform-python-*</code>
* all subpackages will be named <code>platform-python-*</code>
* the debug build will be disabled
* the debug build will be disabled
* the modules that are (before this change) not part of <code>system-python-libs</code> but rather <code>python3-libs</code> will relocate to <code>platform-python-libs-devel</code> to be available only on build-time
* the modules that are (before this change) not part of <code>platform-python-libs</code> but rather <code>python3-libs</code> will relocate to <code>platform-python-libs-devel</code> to be available only on build-time
* <code>platform-python-libs</code> will provide <code>platform-python(abi)</code> instead of <code>python(abi)</code>
* <code>platform-python-libs</code> will provide <code>platform-python(abi)</code> instead of <code>python(abi)</code>


Several packages (see bellow) will be adapted to build a new <code>platform-python-</code> subpackage when built for the platform module. Example specfile of such package:
Several packages (see bellow) will introduce a new <code>platform-python-</code> subpackage. Example specfile of such package:


  %global pypi_name foo
  %global pypi_name foo
Line 89: Line 88:
  Source0:        https://files.pythonhosted.org/packages/source/f/%{pypi_name}/%{pypi_name}-%{version}.tar.gz
  Source0:        https://files.pythonhosted.org/packages/source/f/%{pypi_name}/%{pypi_name}-%{version}.tar.gz
   
   
%if 0%{?platform_module}
  %global pyprefix platform-python
%else
  %global pyprefix python3
  %global platform_module 0
%endif
%if !%{platform_module}
  BuildRequires:  python2-devel
  BuildRequires:  python2-devel
  BuildRequires:  python2-setuptools
  BuildRequires:  python2-setuptools
%endif
   
   
  BuildRequires:  %{pyprefix}-devel
  BuildRequires:  python3-devel
  BuildRequires:  %{pyprefix}-setuptools
  BuildRequires:  python3-setuptools
   
   
BuildRequires:  platform-python-devel
BuildRequires:  platform-python-setuptools
   
   
  BuildArch:      noarch
  BuildArch:      noarch
Line 111: Line 102:
  ...
  ...
   
   
%if !%{platform_module}
  %package -n    python2-%{pypi_name}
  %package -n    python2-%{pypi_name}
  Summary:        %{summary}
  Summary:        %{summary}
Line 119: Line 109:
  %description -n python2-%{pypi_name}
  %description -n python2-%{pypi_name}
  ...
  ...
%endif
   
   
  %package -n    %{pyprefix}-%{pypi_name}
  %package -n    python3-%{pypi_name}
  Summary:        %{summary}
  Summary:        %{summary}
  Requires:      %{pyprefix}-bar
  Requires:      python3-bar
  %{?python_provide:%python_provide %{pyprefix}-%{pypi_name}}
  %{?python_provide:%python_provide python3-%{pypi_name}}
   
   
  %description -n %{pyprefix}-%{pypi_name}
  %description -n python3-%{pypi_name}
  ...
  ...
   
   
%package -n    platform-python-%{pypi_name}
Summary:        %{summary}
Requires:      platform-python-bar
%description -n platform-python-%{pypi_name}
...
   
   
   
   
Line 136: Line 132:
   
   
  %build
  %build
%if !%{platform_module}
  %py2_build
  %py2_build
%endif
  %py3_build
  %py3_build
%plpy_build
   
   
  %install
  %install
%if !%{platform_module}
  %py2_install
  %py2_install
  rm %{buildroot}%{_bindir}/%{pypi_name}
  rm %{buildroot}%{_bindir}/%{pypi_name}
  %endif
  %plpy_install
rm %{buildroot}%{_bindir}/%{pypi_name}
   
   
  %py3_install
  %py3_install
Line 151: Line 147:
   
   
  %check
  %check
%if !%{platform_module}
  export PYTHONPATH=%{buildroot}%{python2_sitelib}
  export PYTHONPATH=%{buildroot}%{python2_sitelib}
  %{__python2} -m unittest discover -v
  %{__python2} -m unittest discover -v
%endif
   
   
  export PYTHONPATH=%{buildroot}%{python3_sitelib}
  export PYTHONPATH=%{buildroot}%{python3_sitelib}
  %{__python3} -m unittest discover -v
  %{__python3} -m unittest discover -v
export PYTHONPATH=%{buildroot}%{platformpython_sitelib}
%{__platformpython} -m unittest discover -v
   
   
   
   
%if !%{platform_module}
  %files -n python2-%{pypi_name}
  %files -n python2-%{pypi_name}
  %doc README.rst CHANGES.rst
  %doc README.rst CHANGES.rst
Line 166: Line 162:
  %{python2_sitelib}/%{pypi_name}.so
  %{python2_sitelib}/%{pypi_name}.so
  %{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
  %{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
%endif
   
   
  %files -n %{pyprefix}-%{pypi_name}
  %files -n python3-%{pypi_name}
  %doc README.rst CHANGES.rst
  %doc README.rst CHANGES.rst
  %license COPYING
  %license COPYING
  %{_bindir}/%{pypi_name}
  %if %{platform_module}
  %exclude %{_bindir}/%{pypi_name}
%else
  %{_bindir}/%{pypi_name}
%endif
  %{python3_sitelib}/%{pypi_name}.cpython-%{python3_version_nodots}*.so
  %{python3_sitelib}/%{pypi_name}.cpython-%{python3_version_nodots}*.so
  %{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
  %{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
   
   
%files -n platform-python-%{pypi_name}
%doc README.rst CHANGES.rst
%license COPYING
%{platformpython_sitelib}/%{pypi_name}.cpython-%{platformplatformpython_version_nodots}*.so
%{platformpython_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
   
   
  %changelog
  %changelog
  ...
  ...


The complete list of affected packages is not final yet. It should be all the Python build dependencies of dnf and their build dependencies as well, excluding packages only needed to build the documentation (because the docs package can be pulled form the standard build).
The complete list of affected packages is not final yet. It should be all the Python build dependencies of dnf and their build dependencies as well, excluding packages only needed to build the documentation.


Currently, we have established this as a list of packages:
Currently, we have established this:


  dbus-python
  dbus-python
Line 237: Line 231:
  pyxdg
  pyxdg
  rpm
  rpm
Only the following list will be installable from the platform module (the rest is merely meant as a build dependency):
platform-python
platform-python-dnf
platform-python-gpg
platform-python-hawkey
platform-python-iniparse
platform-python-libcomps
platform-python-librepo
platform-python-libs
platform-python-rpm
platform-python-six


== Benefit to Fedora ==
== Benefit to Fedora ==


With the Modularity effort a minimal stack of Python only needed to run dnf is needed to be part of the Platform.
With the Modularity effort a minimal stack of Python only needed to run dnf is needed to be part of the Platform. Changing System Python to be a separate stack will allow us to do that.
   
   
    
    
Line 261: Line 242:
* Proposal owners:
* Proposal owners:
** identify a list of runtime and build dependencies of dnf
** identify a list of runtime and build dependencies of dnf
** create a <code>platform-python</code> source package
** create a <code>system-python</code> source package
** change the script that generates the <code>python(abi)</code> requirements to also generate <code>platform-python(abi)</code> requirements
** change the script that generates the <code>python(abi)</code> requirements to also generate <code>system-python(abi)</code> requirements
** revert system-python related things from the {{package|python3}} package, add provides and obsoletes
** revert system-python related things from the {{package|python3}} package
** change specfiles of identified packages to create platform-python- subpackages in platform module
** add a <code>system-python-</code> subpackage to identified list of packages
** switch dnf to use that stack
** switch dnf to use that stack
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
Line 331: Line 312:
If not done on time: Keep dnf running on {{package|python3}}. Finish the work in time of Fedora 28.
If not done on time: Keep dnf running on {{package|python3}}. Finish the work in time of Fedora 28.


If proven not to work as designed: Retire platform-python package and remove all changes in said packages.
If proven not to work as designed: Retire system-python package and remove all the system-python- subpackages. Provide/obsolete system-python- subpackages from their python3- counterparts.


== Documentation ==
== Documentation ==

Revision as of 11:47, 11 July 2017


Platform Python Stack

Summary

A revisit of the System Python change from Fedora 24. Platform Python will be a separate stack fo Python packages aimed to provide all necessary dependencies to run and build dnf. In Fedora 27 this will be Python 3.6 (same version as python3, but this might change in the future with the Modularity effort).

Owner

  • Name: Miro Hrončok
  • Email: <python-maint at redhat.com>
  • Release notes owner:

Current status

  • Targeted release: Fedora 27
  • Last updated: 2017-07-11
  • Tracker bug: <will be assigned by the Wrangler>

Detailed Description

The changes made during the System Python change from Fedora 24 will be reverted.

A new platform-python (source) package will be created as a copy of python3 package. Unlike python3, it will be changed so that:

  • the location of files will not be /usr/lib/python3.6/ but /usr/lib/platform-python/
    • the script that generates the python(abi) requirement will be adapted to generate platform-python(abi) requirement for that location
  • the location of the binary will be /usr/libexec/platform-python
  • all the macros will use the platform python binary and will be renamed
  • all subpackages will be named platform-python-*
  • the debug build will be disabled
  • the modules that are (before this change) not part of platform-python-libs but rather python3-libs will relocate to platform-python-libs-devel to be available only on build-time
  • platform-python-libs will provide platform-python(abi) instead of python(abi)

Several packages (see bellow) will introduce a new platform-python- subpackage. Example specfile of such package:

%global pypi_name foo

Name:           python-%{pypi_name}
Version:        0.13
Release:        1%{?dist}
Summary:        Python %{pypi_name} module

License:        GPLv2
URL:            https://github.com/fedora-python/%{pypi_name}
Source0:        https://files.pythonhosted.org/packages/source/f/%{pypi_name}/%{pypi_name}-%{version}.tar.gz

BuildRequires:  python2-devel
BuildRequires:  python2-setuptools

BuildRequires:  python3-devel
BuildRequires:  python3-setuptools

BuildRequires:  platform-python-devel
BuildRequires:  platform-python-setuptools

BuildArch:      noarch

%description
...

%package -n     python2-%{pypi_name}
Summary:        %{summary}
Requires:       python2-bar
%{?python_provide:%python_provide python2-%{pypi_name}}

%description -n python2-%{pypi_name}
...

%package -n     python3-%{pypi_name}
Summary:        %{summary}
Requires:       python3-bar
%{?python_provide:%python_provide python3-%{pypi_name}}

%description -n python3-%{pypi_name}
...


%package -n     platform-python-%{pypi_name}
Summary:        %{summary}
Requires:       platform-python-bar

%description -n platform-python-%{pypi_name}
...


%prep
%autosetup -n %{pypi_name}-%{version}


%build
%py2_build
%py3_build
%plpy_build

%install
%py2_install
rm %{buildroot}%{_bindir}/%{pypi_name}

%plpy_install
rm %{buildroot}%{_bindir}/%{pypi_name}

%py3_install


%check
export PYTHONPATH=%{buildroot}%{python2_sitelib}
%{__python2} -m unittest discover -v

export PYTHONPATH=%{buildroot}%{python3_sitelib}
%{__python3} -m unittest discover -v

export PYTHONPATH=%{buildroot}%{platformpython_sitelib}
%{__platformpython} -m unittest discover -v


%files -n python2-%{pypi_name}
%doc README.rst CHANGES.rst
%license COPYING
%{python2_sitelib}/%{pypi_name}.so
%{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info

%files -n python3-%{pypi_name}
%doc README.rst CHANGES.rst
%license COPYING
%{_bindir}/%{pypi_name}
%{python3_sitelib}/%{pypi_name}.cpython-%{python3_version_nodots}*.so
%{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info

%files -n platform-python-%{pypi_name}
%doc README.rst CHANGES.rst
%license COPYING
%{platformpython_sitelib}/%{pypi_name}.cpython-%{platformplatformpython_version_nodots}*.so
%{platformpython_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info

%changelog
...

The complete list of affected packages is not final yet. It should be all the Python build dependencies of dnf and their build dependencies as well, excluding packages only needed to build the documentation.

Currently, we have established this:

dbus-python
gpgme
libcomps
libdnf
librepo
pytest
python-argcomplete
python-cffi
python-contextlib2
python-coverage
python-cryptography
python-decorator
python-extras
python-fixtures
python-funcsigs
python-hypothesis
python-idna
python-iniparse
python-jinja2
python-jsonschema
python-keyring
python-linecache2
python-markupsafe
python-mimeparse
python-mock
python-nose
python-pbr
python-pexpect
python-pip
python-ply
python-ptyprocess
python-py
python-pyasn1
python-pycparser
python-pygpgme
python-pytest-timeout
python-rpm-macros
python-SecretStorage
python-setuptools
python-setuptools_scm
python-six
python-testtools
python-traceback2
python-unittest2
python-wheel
pyxattr
pyxdg
rpm

Benefit to Fedora

With the Modularity effort a minimal stack of Python only needed to run dnf is needed to be part of the Platform. Changing System Python to be a separate stack will allow us to do that.


Scope

  • Proposal owners:
    • identify a list of runtime and build dependencies of dnf
    • create a system-python source package
    • change the script that generates the python(abi) requirements to also generate system-python(abi) requirements
    • revert system-python related things from the python3 package
    • add a system-python- subpackage to identified list of packages
    • switch dnf to use that stack
  • Other developers: N/A (not a System Wide Change)
  • Policies and guidelines: N/A (not a System Wide Change)
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

N/A (not a System Wide Change)

User Experience

Nothing should change for the user, except a slightly larger amount of packages will be installed on the system.

Dependencies

N/A (not a System Wide Change)

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? No (not a System Wide Change)
  • Blocks product? No

If not done on time: Keep dnf running on python3. Finish the work in time of Fedora 28.

If proven not to work as designed: Retire system-python package and remove all the system-python- subpackages. Provide/obsolete system-python- subpackages from their python3- counterparts.

Documentation

N/A (not a System Wide Change)

Release Notes