Latest revision as of 18:10, 2 July 2017

AARCH64 - 48-bit VA

Summary

Enable 48bit VA on AARCH64

Owner

Name: Jeremy Linton
Email: jeremy.linton@arm.com
Release notes owner: Simon Clark (sclark)

Current status

Targeted release: Fedora 26
Last updated: 2017-07-02
Tracker bug: #1394837

Detailed Description

The current aarch64 kernel is using a 42-bit process virtual address (VA) space and due to the way aarch64 paging works this limits the max physical address as well. This is fairly limiting for some applications, but aarch64 processors also have support for 48-bit VA's. This is actually 1-bit more than the "47-bit" VA's currently used on x86_64. This change is a fairly minor kernel configuration but it has caused problems with mozjs and luajit based projects because they have stored tags in their pointers in the 48th address bit. The upstream mozjs project has been fixed, and there are a set of changes outstanding for luajit. Futher there, have been patches applied to rawhide to support it (bug #1242326, bug #1375305. bug #1375547), but the packages dependent on js185 require a rebuild due to an ABI change.

Benefit to Fedora

Larger aarch64 processes won't be constrained by both the virtual as well as physical limitations of the 42-bit VA currently in use. This actually also helps with things like hugetlb's and potentially provides a performace boost. Lastly, it allows fedora to boot on a class of machines that have the majority of their RAM higher in the address space.

Scope

Proposal owners:

The kernel config needs to be changed, and the core mozjs/luajit tagging problems need to be resolved. This can either take the form of patching the remaining mozjs packages that have not yet been patched and rebuilding the affected js185 packages, or it can take the form of moving dependent packages onto mozjs versions that have been patched. Futher lua needs to either be updated to the latest mainline or the critical patches need to be applied to the lua versions currently in use.

Other developers:
Release engineering:

Release engineering needs to assure the js185 packages are rebuilt. Peter Robinson has volonteered to assist in this.

List of deliverables:

Policies and guidelines:

N/A

Trademark approval:

N/A (not needed for this Change)

Upgrade/compatibility impact

It shouldn't have any noticable impact on upgrades, currently its possible to boot machines with both 42 and 48 bit VA changes simply by picking diffrent kernel options in grub.

How To Test

Normal system operation on aarch64 hardware. This include running package unit tests/etc as normal and watching out for regressions. The change is system wide because it basically affects the entire system, but should be fairly transparent.

User Experience

Its unlikely a desktop user would notice the change, except possibly it may boot on additional hardware. A server user could potentially utilize more RAM for in-memory databases/etc.

Dependencies

mozjs and luajit packages as well as js's dependent packages.

Need js patches

js-1.8.5 (fixed with js-1.8.5-27.fc27 and js-1.8.5-27.fc26)
mozjs17 (hopefully unnecessary due to polkit move to mozjs24 https://lists.freedesktop.org/archives/polkit-devel/2016-August/000503.html)
mozjs24
mozjs31 (hopefully unnecessary due to 0ad move to mozjs38 http://trac.wildfiregames.com/ticket/3708)
mozjs38
mozjs45

Packages needing rebuild after js-1.8.5 ABI patch

couchdb-0:1.6.1-16
elinks-0:0.12-0.48
erlang-js-0:1.3.0-7 (possible upgrade patch https://github.com/basho/erlang_js/pull/44#issuecomment-247323892)
freewrl-0:3.0.0-1
libEAI-0:3.0.0-1
libproxy-mozjs-0:0.4.12 (hopefully moving to mozjs24)
mediatomb-0:0.12.1-38
pacrunner-0:0.7-7(hopefully moving to mozjs24 or mozjs38)
plowshare-0:2.0.1

Pull lua forward to latest upstream.

Contingency Plan

In the worse case, the kernel can be reverted back to 42-bit VA's with a single configuration change. Otherwise, we can patch additional mozjs versions (mosjz31 if 0ad's patches arn't complete) or drop the dependent packages from the aarch64 disto until they can be fixed (really this only applies to 0ad at this point).

Contingency deadline: Beta freeze
Blocks release? No, the likely problem is an additional package is found to have a problem. I would expect that should this happen it won't be a major bug/package as 48-bit VA's have been in test for a while.

Documentation

As part of this, an attempt has been made to reduce the number of mozjs versions in use. The plan is to move dependent packages to either mozjs24 or mozjs38. There remains some work to move some of the original js 1.8.5 users, but that isn't a critical problem as simply rebuilding them solves the problems caused by the additional address bit.

I can add additional information about the hardware or the dependent projects here if its appropriate.

Release Notes

Enable 48-bit address spaces.

@@ Line 21: / Line 21: @@
 <!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
-= Change Proposal Name <!-- The name of your change proposal --> =
+= AARCH64 - 48-bit VA <!-- The name of your change proposal --> =
 == Summary ==
@@ Line 34: / Line 34: @@
 * Name: [[User:jlinton| Jeremy Linton]]
 * Email: jeremy.linton@arm.com
-* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
+* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> --> [mailto:sclark@fedoraproject.org Simon Clark] ([[User:sclark|sclark]])
 <!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
 * FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
@@ Line 54: / Line 54: @@
 CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 -->
-* Tracker bug: <will be assigned by the Wrangler>
+* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1394837 #1394837]
 == Detailed Description ==
-The current aarch64 kernel is using a 42-bit process virtual address (VA) space. This is fairly limiting, but aarch64 processors also have support for 48-bit VA's. This is actually
+The current aarch64 kernel is using a 42-bit process virtual address (VA) space and due to the way aarch64 paging works this limits the max physical address as well. This is fairly limiting for some applications, but aarch64 processors also have support for 48-bit VA's. This is actually
 -bit more than the "47-bit" VA's currently used on x86_64. This change is a fairly minor kernel configuration but it has caused problems with mozjs and luajit based projects because they have stored tags in their pointers in the 48th address bit. The upstream mozjs project has been fixed, and there are a set of changes outstanding for luajit.  Futher there, have been patches applied to rawhide to support it (bug #1242326, bug #1375305. bug #1375547), but the packages dependent on js185 require a rebuild due to an ABI change.
 == Benefit to Fedora ==
@@ Line 69: / Line 68: @@
 * Proposal owners:
 <!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
+The kernel config needs to be changed, and the core mozjs/luajit tagging problems need to be resolved. This can either take the form of patching the remaining mozjs packages that have not yet been patched and rebuilding the affected js185 packages, or it can take the form of moving dependent packages onto mozjs versions that have been patched. Futher lua needs to either be updated to the latest mainline or the critical patches need to be applied to the lua versions currently in use.
-* Other developers: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Other developers:  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
+* Release engineering:  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-* Release engineering: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuid required?  If a rel-eng ticket exists, add a link here.
 Please work with releng prior to feature submission, and ensure that someone is on board to do any process development work and testing; don't just assume that a bullet point in a change puts someone else on the hook.-->
-** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+Release engineering needs to assure the js185 packages are rebuilt. [[User:Pbrobinson|Peter Robinson]] has volonteered to assist in this.
+* [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
-* Policies and guidelines: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Policies and guidelines: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
+N/A
-* Trademark approval: N/A (not needed for this Change)
+* Trademark approval:
-<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://fedorahosted.org/council/ ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->
+N/A (not needed for this Change)
 == Upgrade/compatibility impact ==
-<!-- What happens to systems that have had a previous versions of Fedora installed and are updated to the version containing this change? Will anything require manual configuration or data migration? Will any existing functionality be no longer supported? -->
+It shouldn't have any noticable impact on upgrades, currently its possible to boot machines with both 42 and 48 bit VA changes simply by picking diffrent kernel options in grub.
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-N/A (not a System Wide Change)
 == How To Test ==
-<!-- This does not need to be a full-fledged document. Describe the dimensions of tests that this change implementation is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.
+Normal system operation on aarch64 hardware.
+This include running package unit tests/etc as normal and watching out for regressions.
+The change is system wide because it basically affects the entire system, but should be fairly transparent.
-Remember that you are writing this how to for interested testers to use to check out your change implementation - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your change.
+== User Experience ==
+Its unlikely a desktop user would notice the change, except possibly it may boot on additional hardware. A server user could potentially utilize more RAM for in-memory databases/etc.
-A good "how to test" should answer these four questions:
+== Dependencies ==
+<!-- What other packages (RPMs) depend on this package?  Are there changes outside the developers' control on which completion of this change depends?  In other words, completion of another change owned by someone else and might cause you to not be able to finish on time or that you would need to coordinate?  Other upstream projects like the kernel (if this is not a kernel change)? -->
-. What special hardware / data / etc. is needed (if any)?
-. How do I prepare my system to test this change? What packages
-need to be installed, config files edited, etc.?
-. What specific actions do I perform to check that the change is
-working like it's supposed to?
-. What are the expected results of those actions?
--->
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-N/A (not a System Wide Change)
+mozjs and luajit packages as well as js's dependent packages.
-== User Experience ==
+Need js patches
-<!-- If this change proposal is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
+*js-1.8.5 (fixed with js-1.8.5-27.fc27 and js-1.8.5-27.fc26)
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+*mozjs17 (hopefully unnecessary due to polkit move to mozjs24 https://lists.freedesktop.org/archives/polkit-devel/2016-August/000503.html)
-N/A (not a System Wide Change)
+*mozjs24
+*mozjs31 (hopefully unnecessary due to 0ad move to mozjs38 http://trac.wildfiregames.com/ticket/3708)
+*mozjs38
+*mozjs45
-== Dependencies ==
+Packages needing rebuild after js-1.8.5 ABI patch
-<!-- What other packages (RPMs) depend on this package?  Are there changes outside the developers' control on which completion of this change depends?  In other words, completion of another change owned by someone else and might cause you to not be able to finish on time or that you would need to coordinate?  Other upstream projects like the kernel (if this is not a kernel change)? -->
+* couchdb-0:1.6.1-16
+* elinks-0:0.12-0.48
+* erlang-js-0:1.3.0-7 (possible upgrade patch https://github.com/basho/erlang_js/pull/44#issuecomment-247323892)
+*freewrl-0:3.0.0-1
+*libEAI-0:3.0.0-1
+*libproxy-mozjs-0:0.4.12 (hopefully moving to mozjs24)
+*mediatomb-0:0.12.1-38
+*pacrunner-0:0.7-7(hopefully moving to mozjs24 or mozjs38)
+*plowshare-0:2.0.1
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+Pull lua forward to latest upstream.
-N/A (not a System Wide Change)
 == Contingency Plan ==
-<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
+In the worse case, the kernel can be reverted back to 42-bit VA's with a single configuration change. Otherwise, we can patch additional mozjs versions (mosjz31 if 0ad's patches arn't complete) or drop the dependent packages from the aarch64 disto until they can be fixed (really this only applies to 0ad at this point).
-* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
-* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Contingency deadline: Beta freeze
 <!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
-* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Blocks release? No, the likely problem is an additional package is found to have a problem. I would expect that should this happen it won't be a major bug/package as 48-bit VA's have been in test for a while.
-* Blocks product? product <!-- Applicable for Changes that blocks specific product release/Fedora.next -->
 == Documentation ==
 <!-- Is there upstream documentation on this change, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->
+As part of this, an attempt has been made to reduce the number of mozjs versions in use. The plan is to move dependent packages to either mozjs24 or mozjs38. There remains some work to move some of the original js 1.8.5 users, but that isn't a critical problem as simply rebuilding them solves the problems caused by the additional address bit.
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+I can add additional information about the hardware or the dependent projects here if its appropriate.
-N/A (not a System Wide Change)
 == Release Notes ==
+Enable 48-bit address spaces.
 <!-- The Fedora Release Notes inform end-users about what is new in the release.  Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ -->
 <!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns.  If there are any such changes involved in this change, indicate them here.  A link to upstream documentation will often satisfy this need.  This information forms the basis of the release notes edited by the documentation team and shipped with the release.
@@ Line 143: / Line 146: @@
 -->
-[[Category:ChangePageIncomplete]]
+[[Category:ChangeAcceptedF26]]
 <!-- When your change proposal page is completed and ready for review and announcement -->
 <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
@@ Line 150: / Line 153: @@
 <!-- Select proper category, default is Self Contained Change -->
-[[Category:SelfContainedChange]]
+[[Category:SystemWideChange]]
-<!-- [[Category:SystemWideChange]] -->

Search

Changes/aarch64-48bitVA: Difference between revisions