From Fedora Project Wiki
mNo edit summary
(add tracker bug and release notes ticket)
 
(14 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= 389_Directory_Server_30 <!-- The name of your change proposal --> =
= 389_Directory_Server_3.0.0 <!-- The name of your change proposal --> =
 
{{Change_Proposal_Banner}}


== Summary ==
== Summary ==
Line 15: Line 13:
This should link to your home wiki page so we know who you are.  
This should link to your home wiki page so we know who you are.  
-->
-->
* Name: [[User:389ds| 389ds Group]]
* Name: 389 Directory Server Development Team
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
* Email: 389-devel (at) lists (dot) fedoraproject (dot) org
* Email: 389-devel (at) lists (dot) fedoraproject (dot) org
Line 27: Line 25:


== Current status ==
== Current status ==
[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF40]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
Line 44: Line 42:
ON_QA -> change is fully code complete
ON_QA -> change is fully code complete
-->
-->
* [<will be assigned by the Wrangler> devel thread]
* [https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/thread/RANL3JMS574QZLM2OJHFGUH62QXOXXZJ/ Announced]
* FESCo issue: <will be assigned by the Wrangler>
* [https://discussion.fedoraproject.org/t/f40-change-proposal-389-directory-server-3-0-0-system-wide/97373 Discussion thread]
* Tracker bug: <will be assigned by the Wrangler>
* FESCo issue: [https://pagure.io/fesco/issue/3120 #3120]
* Release notes tracker: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=2254788 #2254788]
 
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/1063 #1063]


== Detailed Description ==
== Detailed Description ==
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
<!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
Since Fedora 36 (389-ds-base 2.1.0) The 389 ldap directory server supports two kinds of underlying database:
Since Fedora 36 (389-ds-base 2.1.0), the 389 Directory Server supports two kinds of underlying database:
- Berkeley Database (bdb)
 
- Lightning Memory-Mapped Database Manager (lmdb)
* Berkeley Database (BDB)
* Lightning Memory-Mapped Database Manager (LMDB)
 
Newly created instances are still created with BDB by default while libdb is flagged as [[Changes/Libdb deprecated - Fedora Project Wiki|deprecated since Fedora 33]], this change is about to create instances with LMDB by default.


Newly created instances are still created with bdb by default while libdb is flagged as [deprecated since Fedora 33](Changes/Libdb deprecated - Fedora Project Wiki), this change is about to create lmdb instances by default


== Feedback ==
== Feedback ==
Line 64: Line 64:


== Benefit to Fedora ==
== Benefit to Fedora ==
A step on a way to remove a deprecated piece of software no more supported by upstream community.
A step on the way to remove a deprecated piece of software no longer supported by the upstream community.
(See https://fedoraproject.org/wiki/Changes/Libdb_deprecated)
(See https://fedoraproject.org/wiki/Changes/Libdb_deprecated)


== Scope ==
== Scope ==


The change is quite limited in term of development and interface impact
The change is quite limited in terms of development and interface impact, but it impacts a critical component: FreeIPA.
but it impacts a critical component: freeipa.
      
      
[1] Impact on software that uses generic LDAP interface without any specific 389-ds features (like configuration tools and monoitoring entries): No interface impact, but there may be some impacts in term of dynamic (i.e: response time, maximum thoughtput). Especially if browsing (i.e VLV) indexes are used on big lists.
[1] Impact on software that uses generic LDAP interface without any specific 389DS features (like configuration tools and monitoring entries):  
No interface impact, but there may be some impact in terms of dynamic (i.e: response time, maximum throughput). Especially if browsing (i.e VLV) indexes are used on big lists.


[2] Impact on software that explicitly creates 389-ds-base instances (like freeipa) may also have to change the way the database is tuned. (Especially the database maximum size: nsslapd-mdb-max-size)
[2] Impact on software that explicitly creates 389DS instances (like FreeIPA) may also have to change the way the database is tuned. (Especially the database maximum size: `nsslapd-mdb-max-size`)


[3] Impact of software that explicitly uses the backend monitoring LDAP entries:new attributes about lmdb specific data are present.
[3] Impact on software that explicitly uses backend monitoring LDAP entries: new attributes about LMDB specific data are present.
    
    
There are no packages in [3] case and freeipa is the only one in [2] case but there is likely an unknown number of software in [1] case that use the ldap interface without needing a specific LDAP server.
There are no packages in [3] case and FreeIPA is the only one in [2] case but there is likely an unknown number of software in [1] case that use the LDAP interface without needing a specific LDAP server.


So in summary the main risk is to see some regression in tests.
So, in summary, the main risk is to see some regression in tests.


* Proposal owners:
* Proposal owners:
Line 86: Line 86:


Development impact:
Development impact:
- change of a default value of one parameter of the instance creation utility (dscreate)
- provide better visibility of that parameter in: configuration tools (dscreate and dsconf)
- The above changes have already been done to be able to build the test packages
so it remain to merge the commit in the upstream branch and rebase it to rawhide.
- The larger impact is to determine is there are regressions while running 389ds and freeipa test and fix them.
(with the help the help of freeipa team)


These changes have already been done to be able to build the test packages
- Change of a default value of one parameter of the instance creation utility (`dscreate`)
so the only impact is to merge the commit in the upstream branch and rebase it to rawhide.
- Provide a better visibility of that parameter in configuration tools (`dscreate` and `dsconf`)
The larger impact is to determine is there are regressions while running freeipa test and fix them.
 
(with the help the help of freeipa teams)
The above changes have already been done to be able to build the test packages, so the remaining work is to merge the commit in the upstream branch and rebase it to rawhide.
 
The larger impact is to determine if there are any regressions while running 389DS and FreeIPA tests and fix them (with the help of the FreeIPA team).


* Other developers: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Other developers: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
The freeipa team is involved. To help us diagnose the potential freeipa test regressions.


Other Q/E teams may also be involved to test that there is no regression.  
The FreeIPA team is involved to help us diagnose any potential FreeIPA test regressions.
 
Other QA teams may also be involved to test that there are no regressions.  


* Release engineering: [https://pagure.io/releng/issues #Releng issue number] <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Release engineering: [https://pagure.io/releng/issues #Releng issue number] <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
Line 120: Line 117:
<!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->
<!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->


No impact on upgrade because existing instances still use the previously installed backend interface.  
No impact on upgrade because existing instances still use the previously installed backend interface.
 
There should not be any compatibility issues (bugs excepted) with the LDAP requests, but performance may be impacted.
 
There may be issues with applications that explicitly:
 
* Configure 389 Directory Server instances (like FreeIPA) because the set of configuration attributes for the backend depends on the underlying database implementation.
The unused parameters are ignored, so it should not impact the compatibility.
But there is a risk with the new parameters: the "20 GB" default value for the LMDB database maximum size may not be large enough for the application's needs.


There should not have any compatibility issue (bug excepted) with the ldap requests but performances may be impacted.
* Monitor 389 Directory Server backends because the set of attributes in the monitoring entries also depends on the underlying database implementation.


There may be issue with application that explicitly:
More details about these points are described in [https://www.port389.org/docs/389ds/FAQ/Berkeley-DB-deprecation.html 389 Directory Server FAQ - BerkeleyDB backend deprecation]
- configures 389 directory server instances (like freeipa) because the set of configuration attribute for the backend depends of the underlying  database implementation.
  The unused parameter are ignored so it should not impact the compatibility
  But there is a risk for the new parameters: The "20 Gb" default value for the lmdb database maximum size may not be large enough for the application need.
- monitor 389 directory server backends because the the set of attributes in the monitoring entries also depends of the underlying database implementation.
More details about these points are described in [389 Directory Server FAQ - BerkeleyDB backend deprecation](https://www.port389.org/docs/389ds/FAQ/Berkeley-DB-deprecation.html)


== How To Test ==
== How To Test ==
Line 146: Line 146:
-->
-->


a COPR project
[https://copr.fedorainfracloud.org/coprs/g/389ds/389-ds-base-freeipa-tests/ COPR project] contains builds that can be tested (F37, F38, F39, rawhide on x86_64 and s390x).
https://copr.fedorainfracloud.org/coprs/g/389ds/389-ds-base-freeipa-tests/


contains the build that can be tested ( F37, F38, F39, rawhide on x86_64 and s390x )
So you can install them by using:
So you can install them by using:


<PRE>
<pre>
sudo dnf copr enable -y @389ds/389-ds-base-freeipa-tests
sudo dnf copr enable -y @389ds/389-ds-base-freeipa-tests
sudo dnf install 389-ds-base-3.0.0
sudo dnf install 389-ds-base-3.0.0
</PRE>
</pre>


Then you can create new directory server instances and use them.
Then you can create new directory server instances and use them.
Line 170: Line 168:
  - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system.
  - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system.
-->
-->
Prepare the removal of an obsolete piece of software which is no more supported by upstream community and provides more visibility to the replacement.
Prepare for the removal of an obsolete piece of software that is no longer supported by the upstream community. This process aims to provide more visibility to its replacement.


== Dependencies ==
== Dependencies ==
Line 176: Line 174:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
All the dependent packages are owned by 389ds and freeipa teams:  
All the dependent packages are owned by 389DS and FreeIPA teams:  


`dnf repoquery --whatrequires 389-ds-base --recursive`
`dnf repoquery --whatrequires 389-ds-base --recursive`


{| class="wikitable"
{| class="wikitable"
Line 185: Line 183:
! Package !! Owner !! Tests
! Package !! Owner !! Tests
|-
|-
| 389-ds-base-snmp        || 389ds || Tested by 389ds CI test                                                                   |-
| 389-ds-base-snmp        || 389DS || Tested by 389DS CI test
| cockpit-389-ds          || 389ds || Tested by 389ds CI test                                                                   
|-
|-
| freeipa-fas            || freeipa || Tested by freeipa CI test                                                                
| cockpit-389-ds          || 389DS || Tested by 389DS CI test
|-
|-
| freeipa-healthcheck    || freeipa || Tested by freeipa CI test                                                              
| freeipa-fas            || FreeIPA || Tested by FreeIPA CI test
|-
|-
| freeipa-server          || freeipa || Tested by freeipa CI test
| freeipa-healthcheck    || FreeIPA || Tested by FreeIPA CI test
|-
|-
| freeipa-server-dns      || freeipa || Tested by freeipa CI test                                                                
| freeipa-server         || FreeIPA || Tested by FreeIPA CI test
|-
|-
| freeipa-server-trust-ad || freeipa || Tested by freeipa CI test
| freeipa-server-dns      || FreeIPA || Tested by FreeIPA CI test
|-
|-
| migrationtools          || 389ds || Not impacted by the change
| freeipa-server-trust-ad || FreeIPA || Tested by FreeIPA CI test
|-
|-
| slapi-nis              || freeipa || Tested by freeipa CI test
| migrationtools          || 389DS || Not impacted by the change
|-
| slapi-nis              || FreeIPA || Tested by FreeIPA CI test
|}
|}


Line 206: Line 205:


<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Contingency mechanism: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
Contingency mechanism:
- Revert the default backend implementation to Berkeley Database
- if libdb-5.3/libdb-devel are no more delivered: build it locally and link it as a static libraries. (It will probably be done anyway to handle the db migration)
 
Contingency deadline: Fedora 40 branching from Rawhide
Blocks release? No


* Revert the default backend implementation to Berkeley Database
* If libdb-5.3 or libdb-devel is no longer shipped in Fedora:
The plan is to build libdb static library as part of 389-ds-base build process and link libback-ldbm statically with it
(as described in https://docs.fedoraproject.org/en-US/packaging-guidelines/#_statically_linking_executables).
The risk is that we may have to generate patches in case of CVE that impacts 389-ds or in case of build environment changes until we can remove the support of the bdb backend. libdb-5.3.28-55.fc38.src shows that the risk is quite limited (1 cve and one set of patches related to c99 since 2017).
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
* Contingency deadline: The beta freeze (2024-02-20)
* Contingency deadline: Fedora 40 beta freeze (2024-02-20)
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
* Blocks release? No
* Blocks release? No
Line 224: Line 222:
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->


389 Directory Server FAQ - BerkeleyDB backend deprecation: https://www.port389.org/docs/389ds/FAQ/Berkeley-DB-deprecation.html
[https://www.port389.org/docs/389ds/FAQ/Berkeley-DB-deprecation.html Directory Server FAQ - BerkeleyDB backend deprecation]


== Release Notes ==
== Release Notes ==
Line 232: Line 230:
Release Notes are not required for initial draft of the Change Proposal but has to be completed by the Change Freeze.  
Release Notes are not required for initial draft of the Change Proposal but has to be completed by the Change Freeze.  
-->
-->
Need to add write release notes for 3.0.0  
Need to write release notes for 3.0.0 in https://www.port389.org/docs/389ds/releases/release-notes.html as usual for 389DS.
in https://www.port389.org/docs/389ds/releases/release-notes.html as usual for 389ds.

Latest revision as of 00:38, 16 December 2023

389_Directory_Server_3.0.0

Summary

389-ds-base upgrade from version 2.4.4 to the latest upstream version 3.0.0 in Fedora. Newly created instances now are using LDMB database by default instead of BerkeleyDB.


Owner

  • Name: 389 Directory Server Development Team
  • Email: 389-devel (at) lists (dot) fedoraproject (dot) org

Primary contact:


Current status

Detailed Description

Since Fedora 36 (389-ds-base 2.1.0), the 389 Directory Server supports two kinds of underlying database:

  • Berkeley Database (BDB)
  • Lightning Memory-Mapped Database Manager (LMDB)

Newly created instances are still created with BDB by default while libdb is flagged as deprecated since Fedora 33, this change is about to create instances with LMDB by default.


Feedback

No feedback yet.

Benefit to Fedora

A step on the way to remove a deprecated piece of software no longer supported by the upstream community. (See https://fedoraproject.org/wiki/Changes/Libdb_deprecated)

Scope

The change is quite limited in terms of development and interface impact, but it impacts a critical component: FreeIPA.

[1] Impact on software that uses generic LDAP interface without any specific 389DS features (like configuration tools and monitoring entries): No interface impact, but there may be some impact in terms of dynamic (i.e: response time, maximum throughput). Especially if browsing (i.e VLV) indexes are used on big lists.

[2] Impact on software that explicitly creates 389DS instances (like FreeIPA) may also have to change the way the database is tuned. (Especially the database maximum size: nsslapd-mdb-max-size)

[3] Impact on software that explicitly uses backend monitoring LDAP entries: new attributes about LMDB specific data are present.

There are no packages in [3] case and FreeIPA is the only one in [2] case but there is likely an unknown number of software in [1] case that use the LDAP interface without needing a specific LDAP server.

So, in summary, the main risk is to see some regression in tests.

  • Proposal owners:

Development impact:

- Change of a default value of one parameter of the instance creation utility (dscreate) - Provide a better visibility of that parameter in configuration tools (dscreate and dsconf)

The above changes have already been done to be able to build the test packages, so the remaining work is to merge the commit in the upstream branch and rebase it to rawhide.

The larger impact is to determine if there are any regressions while running 389DS and FreeIPA tests and fix them (with the help of the FreeIPA team).

  • Other developers:

The FreeIPA team is involved to help us diagnose any potential FreeIPA test regressions.

Other QA teams may also be involved to test that there are no regressions.

N/A (not needed for this Change): No strong coordination is needed as only a single group of package is directly impacted.

  • Policies and guidelines: N/A (not needed for this Change)
  • Trademark approval: N/A (not needed for this Change)
  • Alignment with Community Initiatives:

Upgrade/compatibility impact

No impact on upgrade because existing instances still use the previously installed backend interface.

There should not be any compatibility issues (bugs excepted) with the LDAP requests, but performance may be impacted.

There may be issues with applications that explicitly:

  • Configure 389 Directory Server instances (like FreeIPA) because the set of configuration attributes for the backend depends on the underlying database implementation.

The unused parameters are ignored, so it should not impact the compatibility. But there is a risk with the new parameters: the "20 GB" default value for the LMDB database maximum size may not be large enough for the application's needs.

  • Monitor 389 Directory Server backends because the set of attributes in the monitoring entries also depends on the underlying database implementation.

More details about these points are described in 389 Directory Server FAQ - BerkeleyDB backend deprecation

How To Test

COPR project contains builds that can be tested (F37, F38, F39, rawhide on x86_64 and s390x).

So you can install them by using: 

sudo dnf copr enable -y @389ds/389-ds-base-freeipa-tests
sudo dnf install 389-ds-base-3.0.0

Then you can create new directory server instances and use them.

User Experience

Prepare for the removal of an obsolete piece of software that is no longer supported by the upstream community. This process aims to provide more visibility to its replacement.

Dependencies

All the dependent packages are owned by 389DS and FreeIPA teams:

dnf repoquery --whatrequires 389-ds-base --recursive

Package Owner Tests
389-ds-base-snmp 389DS Tested by 389DS CI test
cockpit-389-ds 389DS Tested by 389DS CI test
freeipa-fas FreeIPA Tested by FreeIPA CI test
freeipa-healthcheck FreeIPA Tested by FreeIPA CI test
freeipa-server FreeIPA Tested by FreeIPA CI test
freeipa-server-dns FreeIPA Tested by FreeIPA CI test
freeipa-server-trust-ad FreeIPA Tested by FreeIPA CI test
migrationtools 389DS Not impacted by the change
slapi-nis FreeIPA Tested by FreeIPA CI test

Contingency Plan

  • Contingency mechanism:
  • Revert the default backend implementation to Berkeley Database
  • If libdb-5.3 or libdb-devel is no longer shipped in Fedora:

The plan is to build libdb static library as part of 389-ds-base build process and link libback-ldbm statically with it (as described in https://docs.fedoraproject.org/en-US/packaging-guidelines/#_statically_linking_executables). The risk is that we may have to generate patches in case of CVE that impacts 389-ds or in case of build environment changes until we can remove the support of the bdb backend. libdb-5.3.28-55.fc38.src shows that the risk is quite limited (1 cve and one set of patches related to c99 since 2017).

  • Contingency deadline: Fedora 40 beta freeze (2024-02-20)
  • Blocks release? No

Documentation

Directory Server FAQ - BerkeleyDB backend deprecation

Release Notes

Need to write release notes for 3.0.0 in https://www.port389.org/docs/389ds/releases/release-notes.html as usual for 389DS.

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/389_Directory_Server_3.0.0&oldid=696670"