From Fedora Project Wiki
 
(23 intermediate revisions by 2 users not shown)
Line 2: Line 2:


= Deprecate python-toml =
= Deprecate python-toml =
{{Change_Proposal_Banner}}


== Summary ==
== Summary ==
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. Note that motivation for the change should be in the Benefit to Fedora section below, and this part should answer the question "What?" rather than "Why?". -->
<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. Note that motivation for the change should be in the Benefit to Fedora section below, and this part should answer the question "What?" rather than "Why?". -->


The {{package|python-toml}} ({{package|python3-toml}}) package will be [https://docs.fedoraproject.org/en-US/packaging-guidelines/deprecating-packages/ deprecated] in [[Releases/38|Fedora 38]]. The [https://pypi.org/project/toml/ upstream package is considered dead] and Python 3.11 contains [https://peps.python.org/pep-0680/ a toml-reading library in the standard library]. Existing Fedora packages depend on {{package|python3-toml}}, so we cannot remove it yet. Packagers are encouraged to work with upstream to switch to [https://peps.python.org/pep-0680/ tomllib]/[https://pypi.org/project/tomli/ tomli] for reading toml or [https://pypi.org/project/tomli/ tomli-w] for writing it. But {{package|python-toml}} remains available until it is a leaf package, it will be removed then (possibly not yet in Fedora 38).
The {{package|python-toml}} (`python3-toml`) package will be [https://docs.fedoraproject.org/en-US/packaging-guidelines/deprecating-packages/ deprecated] in [[Releases/38|Fedora 38]]. The [https://pypi.org/project/toml/ upstream toml package] is considered unmaintained (see [[#Detailed_Description|description]]) and Python 3.11 contains [https://peps.python.org/pep-0680/ a TOML-reading library in the standard library]. Existing Fedora packages depend on {{package|python-toml}}, so we cannot remove it yet. Packagers are encouraged to work with upstreams to switch to [https://peps.python.org/pep-0680/ tomllib]/[https://pypi.org/project/tomli/ tomli] for reading toml or [https://pypi.org/project/tomli/ tomli-w] for writing it. But {{package|python-toml}} remains available until it is a leaf package, it will be removed then (possibly not yet in Fedora 38).


== Owner ==
== Owner ==
Line 23: Line 21:


== Current status ==
== Current status ==
[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF38]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
Line 41: Line 39:
ON_QA -> change is fully code complete
ON_QA -> change is fully code complete
-->
-->
* FESCo issue: <will be assigned by the Wrangler>
* [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/A2IPHXETVWOCWJUJBHKUZO2WODZXIYVT/ devel thread]
* Tracker bug: <will be assigned by the Wrangler>
* FESCo issue: [https://pagure.io/fesco/issue/2879 #2879]
* Release notes tracker: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=2137375 #2137375]
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/891 #891]


== Detailed Description ==
== Detailed Description ==
Line 49: Line 48:
The {{package|python-toml}} package is [https://pypi.org/project/toml/ unmaintained upstream]. It does not support the latest TOML standard and no longer releases newer versions.
The {{package|python-toml}} package is [https://pypi.org/project/toml/ unmaintained upstream]. It does not support the latest TOML standard and no longer releases newer versions.


We'd like to drop it from Fedora, but several packages still require it. Before we attempt to remove the package, we need to stop new packages to (Build)Require {{package|python3-toml}}, hence we want to have it [https://docs.fedoraproject.org/en-US/packaging-guidelines/deprecating-packages/ deprecated]. No new packages that require it may be added to Fedora and existing packages may not gain new dependencies on it. Requiring it only `if python3 < 3.11` or similar is allowed because python3 on Fedora 38 is 3.11.
We'd like to drop it from Fedora, but several packages still require it. Before we attempt to remove the package, we need to stop new packages to (Build)Require `python3-toml`, hence we want to have it [https://docs.fedoraproject.org/en-US/packaging-guidelines/deprecating-packages/ deprecated]. No new packages that require it may be added to Fedora and existing packages may not gain new dependencies on it. Requiring it only `if python3 < 3.11` or similar is allowed because python3 on Fedora 38 is 3.11.


Packagers are encouraged to switch to an alternative toml library with upstream involvement. Downstream-only patches to switch are not encouraged. Change owners recommend the following alternatives:
Packagers are encouraged to switch to an alternative TOML library with upstream involvement. Downstream-only patches to switch are not encouraged. Change owners recommend the following alternatives:


* Use the [https://docs.python.org/3.11/library/tomllib.html tomllib] module from the standard library to read TOML with Python 3.11+.
* Use the [https://docs.python.org/3.11/library/tomllib.html tomllib] module from the standard library to read TOML with Python 3.11+.
* Use the {{package|python-tomli}} package to read TOML with an older version of Python. The tomllib module has started as tomli and they share the same API except for the module name.
* Use the {{package|python-tomli}} package to read TOML with an older version of Python. The `tomllib` module has started as `tomli` and they share the same API except for the module name.
* Use the {{package|python-tomli-w}} package to write TOML.
* Use the {{package|python-tomli-w}} package to write TOML.


Note that <code>repoquery</code> gives many packages depending on {{package|python3-toml}}:
Note that <code>repoquery</code> gives many packages depending on `python3-toml`:


  $ repoquery --repo=rawhide{,-source} --whatrequires python3-toml | wc -l
  $ repoquery --repo=rawhide{,-source} --whatrequires python3-toml | wc -l
Line 154: Line 153:
* {{package|cvc4}}
* {{package|cvc4}}
* {{package|fedora-license-data}} (false positive, dependency conditional on python3 < 3.11)
* {{package|fedora-license-data}} (false positive, dependency conditional on python3 < 3.11)
* {{package|fedora-messaging}}
* {{package|fedora-messaging}} ([https://github.com/fedora-infra/fedora-messaging/pull/274 upstream fix])
* {{package|gi-docgen}}
* {{package|gi-docgen}} ([https://gitlab.gnome.org/GNOME/gi-docgen/-/merge_requests/168 merged upstream PR])
* {{package|jrnl}}
* {{package|jrnl}}
* {{package|micropipenv}} (already fixed in the meantime)
* {{package|micropipenv}} (already fixed in the meantime)
Line 168: Line 167:
* {{package|python-botocore}}
* {{package|python-botocore}}
* {{package|python-box}}
* {{package|python-box}}
* {{package|python-build}
* {{package|python-build}}
* {{package|python-deepdiff}}
* {{package|python-deepdiff}}
* {{package|python-devicely}}
* {{package|python-devicely}}
Line 177: Line 176:
* {{package|python-check-manifest}}
* {{package|python-check-manifest}}
* {{package|python-interrogate}}
* {{package|python-interrogate}}
* {{package|python-jaraco-functools}}
* {{package|python-jaraco-functools}} (superfluous, [https://src.fedoraproject.org/rpms/python-jaraco-functools/pull-request/5 fixed])
* {{package|python-jinja2-cli}}
* {{package|python-jinja2-cli}}
* {{package|python-jsonpickle}}
* {{package|python-jsonpickle}}
Line 192: Line 191:
* {{package|python-pyedflib}}
* {{package|python-pyedflib}}
* {{package|python-pyqt-feedback-flow}}
* {{package|python-pyqt-feedback-flow}}
* {{package|python-rich}}
* {{package|python-rich}} (superfluous, [https://src.fedoraproject.org/rpms/python-rich/pull-request/11 proposed fix])
* {{package|python-rst-linker}}
* {{package|python-rst-linker}}
* {{package|python-sklearn-nature-inspired-algorithms}}
* {{package|python-sklearn-nature-inspired-algorithms}}
Line 208: Line 207:


=== Migrating to tomllib ===
=== Migrating to tomllib ===
{{admon/important|Migrating to `tomllib`-only is only possible if the code will be run on Python 3.11 or newer. See the [[#Migrating_to_tomllib_on_Python_3.11+_and_falling_back_to_tomli]] section for a more portable solution.}}


* Remove any `toml` requirements from upstream metadata.
* Remove any `toml` requirements from upstream metadata.
Line 215: Line 216:


=== Migrating to tomli ===
=== Migrating to tomli ===
{{admon/important|Migrating to `tomli`-only leaves an unnecessary dependency on a 3rd party library which is now essentially a backport of a standard library module. See the [[#Migrating_to_tomllib_on_Python_3.11+_and_falling_back_to_tomli]] section for a more sustainable solution.}}


* Change any `toml` requirements in upstream metadata to `tomli`.
* Change any `toml` requirements in upstream metadata to `tomli`.
Line 242: Line 245:
     import tomllib
     import tomllib


Upstream example: https://github.com/pytest-dev/pytest/pull/9741
Upstream examples: https://github.com/fedora-infra/fedora-messaging/pull/274 or https://github.com/pytest-dev/pytest/pull/9741


A more complex example that also falls back to `toml` on Python 2.7 or 3.6: https://github.com/tox-dev/tox/pull/2463
A more complex example that also falls back to `toml` on Python 2.7 or 3.6: https://github.com/tox-dev/tox/pull/2463
Line 290: Line 293:


== Scope ==
== Scope ==
* Proposal owners: Deprecate {{package|python3-toml}}. Work with packagers and upstream developers to remove the dependency. Monitor the remaining dependent packages and eventually retire {{package|python-toml}} (unlikely in Fedora 38).
* Proposal owners: Deprecate `python3-toml`. Work with packagers and upstream developers to remove the dependency. Monitor the remaining dependent packages and eventually retire {{package|python-toml}} (unlikely in Fedora 38).
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->


* Other developers: No action needed. Don't add new dependencies on {{package|python3-toml}}. <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Other developers: No action needed. Don't add new dependencies on `python3-toml`. <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->


Line 306: Line 309:
<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://pagure.io/Fedora-Council/tickets/issues ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->
<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://pagure.io/Fedora-Council/tickets/issues ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->


* Alignment with Objectives: Minimization (the plan is to have one less Python TOML library in Fedora and RHEL 10+)
* Alignment with Objectives:  
<!-- Does your proposal align with the current Fedora Objectives: https://docs.fedoraproject.org/en-US/project/objectives/ ? It's okay if it doesn't, but it's something to consider -->


== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==
<!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->
<!-- What happens to systems that have had a previous version of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
The package will remain available. Only new packages cannot depend on it.
The package will remain available. Only new packages cannot depend on it.
Once retired, we don't plan to provide <code>python3-toml</code> from {{package|python3-libs}} or {{package|python3-tomli}}, because it cannot work as drop-in replacement (the Python module has a different name and sligthly different API). The package will eventually be obsoleted by {{package|fedora-obsolete-packages}} once retired, but that is unlikely to happen soon.
Once retired, we don't plan to provide <code>python3-toml</code> from `python3-libs` or `python3-tomli`, because it cannot work as a drop-in replacement (the Python module has a different name and slightly different API). The package will eventually be obsoleted by {{package|fedora-obsolete-packages}} once retired, but that is unlikely to happen soon.


== How To Test ==
== How To Test ==

Latest revision as of 18:10, 29 December 2022


Deprecate python-toml

Summary

The python-toml (python3-toml) package will be deprecated in Fedora 38. The upstream toml package is considered unmaintained (see description) and Python 3.11 contains a TOML-reading library in the standard library. Existing Fedora packages depend on python-toml, so we cannot remove it yet. Packagers are encouraged to work with upstreams to switch to tomllib/tomli for reading toml or tomli-w for writing it. But python-toml remains available until it is a leaf package, it will be removed then (possibly not yet in Fedora 38).

Owner

Current status

Detailed Description

The python-toml package is unmaintained upstream. It does not support the latest TOML standard and no longer releases newer versions.

We'd like to drop it from Fedora, but several packages still require it. Before we attempt to remove the package, we need to stop new packages to (Build)Require python3-toml, hence we want to have it deprecated. No new packages that require it may be added to Fedora and existing packages may not gain new dependencies on it. Requiring it only if python3 < 3.11 or similar is allowed because python3 on Fedora 38 is 3.11.

Packagers are encouraged to switch to an alternative TOML library with upstream involvement. Downstream-only patches to switch are not encouraged. Change owners recommend the following alternatives:

  • Use the tomllib module from the standard library to read TOML with Python 3.11+.
  • Use the python-tomli package to read TOML with an older version of Python. The tomllib module has started as tomli and they share the same API except for the module name.
  • Use the python-tomli-w package to write TOML.

Note that repoquery gives many packages depending on python3-toml: 

$ repoquery --repo=rawhide{,-source} --whatrequires python3-toml | wc -l
443

This is because many packages BuildRequire (python3dist(toml) if python3-devel < 3.11) due to pyproject-rpm-macros. 

$ repoquery --repo=rawhide{,-source} --whatrequires '(python3dist(toml) if python3-devel < 3.11)' | wc -l
413

The change owners don't know how to easily filter them out, but when filtered the hard way, this remains:

(Results from 2022-10-05, may contain false positives.) 

$ for pkg in $(repoquery --repo=rawhide{,-source} --whatrequires python3-toml); do repoquery -q --repo=rawhide{,-source} --requires $pkg | grep -Fv '(python3dist(toml) if python3-devel < 3.11)' | grep -Eq '(\(|-)toml\b' && echo $pkg; done
academic-admin-0:0.5.1-10.fc37.noarch
academic-admin-0:0.5.1-10.fc37.src
bandit-0:1.7.4-3.fc37.src
bst-external-0:0.29.0-1.fc38.src
cvc4-0:1.8-12.fc37.src
fedora-license-data-0:1.5-1.fc38.src
fedora-messaging-0:3.1.0-5.fc38.src
gi-docgen-0:2022.1-7.fc38.noarch
gi-docgen-0:2022.1-7.fc38.src
jrnl-0:3.0-3.fc37.src
micropipenv-0:1.4.2-1.fc37.noarch
pre-commit-0:2.20.0-2.fc37.noarch
pre-commit-0:2.20.0-2.fc37.src
pylint-0:2.14.4-3.fc37.src
python-anyconfig-0:0.13.0-3.fc37.src
python-anymarkup-0:0.8.1-10.fc37.src
python-anymarkup-core-0:0.8.1-9.fc37.src
python-ast-monitor-0:0.2.1-1.fc38.src
python-asttokens-0:2.0.8-1.fc38.src
python-autopep8-0:1.6.0-5.fc37.src
python-botocore-0:1.27.86-1.fc38.src
python-box-0:6.0.2-1.fc38.src
python-build-0:0.8.0-4.fc37.src
python-check-manifest-0:0.48-3.fc37.src
python-deepdiff-0:5.8.2-2.fc37.src
python-devicely-0:1.1.1-3.fc37.src
python-elpy-0:1.34.0-8.fc37.src
python-exoscale-0:0.7.1-4.fc37.src
python-fasjson-client-0:1.0.7-5.fc38.src
python-fireflyalgorithm-0:0.3.2-2.fc37.src
python-interrogate-0:1.5.0-4.fc37.src
python-jsonpickle-0:2.2.0-4.fc37.src
python-lsp-black-0:1.2.0-3.fc37.src
python-matrix-nio-0:0.19.0-6.fc38.src
python-molecule-podman-0:1.0.1-4.fc37.src
python-neurom-0:3.1.0-5.fc37.src
python-niaaml-0:1.1.11-1.fc38.src
python-niaarm-0:0.2.1-2.fc38.src
python-niaclass-0:0.1.2-8.fc37.src
python-nikola-0:8.2.2-4.fc37.src
python-pendulum-0:2.1.2-8.fc37.src
python-podman-3:4.2.0-7.fc38.src
python-pyedflib-0:0.1.30-2.fc37.src
python-pyqt-feedback-flow-0:0.1.6-3.fc37.src
python-rich-0:12.6.0-1.fc38.src
python-rst-linker-0:2.3.1-1.fc38.src
python-sklearn-nature-inspired-algorithms-0:0.9.0-3.fc37.src
python-sport-activities-features-0:0.3.6-1.fc38.src
python-stochastic-0:0.7.0-2.fc37.src
python-toml-adapt-0:0.2.7-3.fc37.src
python-usort-0:0.6.3-7.fc37.src
python-vulture-0:2.6-1.fc38.src
python3-anymarkup-0:0.8.1-10.fc37.noarch
python3-autopep8-0:1.6.0-5.fc37.noarch
python3-box-0:6.0.2-1.fc38.noarch
python3-exoscale-0:0.7.1-4.fc37.noarch
python3-fasjson-client-0:1.0.7-5.fc38.noarch
python3-fedora-messaging-0:3.1.0-5.fc38.noarch
python3-interrogate-0:1.5.0-4.fc37.noarch
python3-jaraco-functools-0:3.5.2-1.fc38.noarch
python3-jinja2-cli-0:0.8.2-3.fc37.noarch
python3-lsp-black-0:1.2.0-3.fc37.noarch
python3-nikola-0:8.2.2-4.fc37.noarch
python3-podman-3:4.2.0-7.fc38.noarch
python3-sklearn-nature-inspired-algorithms-0:0.9.0-3.fc37.noarch
python3-toml-adapt-0:0.2.7-3.fc37.noarch
python3-usort-0:0.6.3-7.fc37.noarch
python3-vulture-0:2.6-1.fc38.noarch
rpmlint-0:2.2.0-7.fc38.noarch
rpmlint-0:2.2.0-7.fc38.src
sip6-0:6.6.2-2.fc37.src
sip6-0:6.6.2-2.fc37.x86_64
teampulls-0:0.2.2-10.fc37.noarch
trac-tracnav-plugin-0:4.3-7.fc37.src

List of components still (Build)Requiring python3-toml

Once all dependencies are removed, we plan to retire python-toml, whether it will be in Fedora 38 (unlikely) or later.

Migrating to tomllib

Migrating to tomllib-only is only possible if the code will be run on Python 3.11 or newer. See the #Migrating_to_tomllib_on_Python_3.11+_and_falling_back_to_tomli section for a more portable solution.
  • Remove any toml requirements from upstream metadata.
  • Change all toml imports to tomllib imports.
  • Change all references of TomlDecodeError to TOMLDecodeError.
  • Open files in binary mode, if passing file objects to tomllib.load().

Migrating to tomli

Migrating to tomli-only leaves an unnecessary dependency on a 3rd party library which is now essentially a backport of a standard library module. See the #Migrating_to_tomllib_on_Python_3.11+_and_falling_back_to_tomli section for a more sustainable solution.
  • Change any toml requirements in upstream metadata to tomli.
  • Change all toml imports to tomli imports.
  • Change all references of TomlDecodeError to TOMLDecodeError.
  • Open files in binary mode, if passing file objects to tomli.load().

Migrating to tomllib on Python 3.11+ and falling back to tomli

  • Change any toml requirements in upstream metadata to tomli;python_version<"3.11".
  • Change all toml imports to sys.version_info-conditional or try: except ImportError: tomli/tomllib imports.
  • Change all references of TomlDecodeError to TOMLDecodeError.
  • Open files in binary mode, if passing file objects to tomllib.load().

Example of importing: 

try:
    import tomllib
except ImportError:
    import tomli as tomllib

Or: 

if sys.version_info < (3, 11):
    import tomli as tomllib
else:
    import tomllib

Upstream examples: https://github.com/fedora-infra/fedora-messaging/pull/274 or https://github.com/pytest-dev/pytest/pull/9741

A more complex example that also falls back to toml on Python 2.7 or 3.6: https://github.com/tox-dev/tox/pull/2463

A more complex example that supports pytoml, toml, tomli and tomllib: https://github.com/thoth-station/micropipenv/pull/241

Migrating to tomli-w

  • Change any toml requirements in upstream metadata to tomli-w.
  • Change all toml imports to tomli_w imports.
  • Open files in binary mode, if passing file objects to tomli_w.dump().

A more complex example that migrates to tomli, tomllib and tomli-w: https://github.com/rpm-software-management/rpmlint/pull/905

Feedback

Benefit to Fedora

An upstream dead package will not be depended upon by new packages.

Scope

  • Proposal owners: Deprecate python3-toml. Work with packagers and upstream developers to remove the dependency. Monitor the remaining dependent packages and eventually retire python-toml (unlikely in Fedora 38).
  • Other developers: No action needed. Don't add new dependencies on python3-toml.
  • Release engineering: N/A (not a System Wide Change)
  • Policies and guidelines: N/A (not needed for this Change)
  • Trademark approval: N/A (not needed for this Change)
  • Alignment with Objectives:

Upgrade/compatibility impact

The package will remain available. Only new packages cannot depend on it. Once retired, we don't plan to provide python3-toml from python3-libs or python3-tomli, because it cannot work as a drop-in replacement (the Python module has a different name and slightly different API). The package will eventually be obsoleted by fedora-obsolete-packages once retired, but that is unlikely to happen soon.

How To Test

$ repoquery --repo=rawhide --provides python3-toml
...
deprecated()
...

User Experience

No changes.

Dependencies

N/A (not a System Wide Change)

Contingency Plan

  • Contingency mechanism: revert the deprecation
  • Contingency deadline: Final Freeze
  • Blocks release? No

Documentation

N/A (not a System Wide Change)

Release Notes

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/DeprecatePythonToml&oldid=664460"