From Fedora Project Wiki
 
(38 intermediate revisions by the same user not shown)
Line 37: Line 37:
* Support for developing new enclaves
* Support for developing new enclaves
** Header files for the enclave code (a minimalist C library, C++ library, crypto and some other misc libraries)
** Header files for the enclave code (a minimalist C library, C++ library, crypto and some other misc libraries)
** Static library archives for the enclave code.  
** Static library archives for linking into the enclave binaries.  
** Build helper tools (for signing enclaves, generating code enclave API entrypoints)
** Build helper tools (for signing enclaves, generating code enclave API entrypoints)
* Support for developing applications that use enclaves
* Support for developing applications that use enclaves, eg to be able to load and communicate with enclaves.
** Header files for platform code
** Header files for platform code
** Dynamic libraries for platform code
** Dynamic libraries for platform code
Line 47: Line 47:
** Quote generation daemon - assists QEMU in acquiring signed attestation reports for TDX VMs.
** Quote generation daemon - assists QEMU in acquiring signed attestation reports for TDX VMs.


=== Enclaves vs packaging & licensing guidelines ===
The code and binaries related to the host OS platform components will be installed in the normal filesystem locations common to all Fedora packages. e.g. libraries in '''/usr/lib64''', headers in '''/usr/include''' and binaries in '''/usr/bin'''.


Generally Fedora requires all software to be fully built from source, with the complete & corresponding source made available under one of the permitted code licenses.
For the purposes of packaging, the enclaves will be treated as cross-compilation target. While the compiler build architecture target is '''x86_64''', the runtime has custom C / C++ libraries that must be used, and requires a separate code loader. Enclaves cannot be directly linked to applications, they are strictly independent libraries.


There is, however, an exception for firmware binaries which do not have to provide source, and merely required to permit free redistribution.
With this in mind, all enclave related headers and libraries are proposed to be installed at '''/usr/x86_64-intel-sgx''', specifically under ''lib64'' and ''include'' sub-directories. There are no binaries for enclaves, only libraries, so no '''bin''' dir is required.


The question is how to treat enclaves, and deciding this requires understanding what SGX enclaves are and how they are used.
The generated binary packages will generally all have an 'sgx-' prefix to their name, with enclave related packages having an more specialized 'sgx-enclave-' name prefix.


==== Enclaves as a new platform build target ====
== Feedback ==


SGX enclaves are compiled to x86 machine code from C/C++ source, using custom C/C++ runtime libraries, instead of the host OS glibc / libstdc++. IOW, while the enclaves can build with the existing x86 C/C++ toolchain, they are more like a cross-compilation target in the way that a separate non-Linux runtime is used.
* '''Objection:''' ''The SGX technology can be used as a way to implement DRM. Notable example has been BluRay playback.''. '''Answer:''' As with many technologies, it is possible to use SGX in ways that are both positive and negative, wrt the owner / users of a machine. Use of SGX for DRM in BluRay playback is hostile to the owner/user of a machine. This change is '''NOT''' proposing to introduce / support any such usage/applications in Fedora. The fact that bad uses of SGX exist outside of Fedora, must not block the use of SGX in in Fedora for scenarios where it can offer features that benefit Fedora's users.


The resulting SGX enclave binary uses the ELF format, creating ELF executables, albeit with a '''.so''' file suffix. While the executable binary does have a '''.interp''' ELF section, the referenced '''ld-linux.so''' cannot load an enclave. A specialized SGX host OS library must be called to create an enclave and load code into it. Given that, the '''.interp''' ELF section is in fact redundant and misleading, and has [https://github.com/intel/linux-sgx/pull/1061 been proposed for removal].
* '''Objection:''' ''The SGX enclave code is not open source, because it requires a vendor signature on output binaries''. '''Answer:''' 100% of the SGX code is made available under a variety open source licenses (Apache, BSD, MIT, GPL & more). One of the architectural enclaves, '''pce''', requires an Intel signature because it is used to establish the root of trust with the hardware. In their role of bootstrapping use of SGX hardware, the architectural enclaves are a type of firmware, and it is normal for firmware to have a vendor signature. In contrast to almost all firmware which is proprietary, the architectural enclave code is all under Fedora approved OSS licenses. All of the architectural enclaves, can be rebuilt from source by a user, and signed with a user specified key if desired. If doing so, the limitation is that if the '''pce''' enclave is not signed by an Intel key, it will not be possible to obtain a certificate for the host from Intel's public PCS API service, and thus the integrity of SGX signed quotes cannot be validated back to the Intel root CA certificate.


To load an SGX enclave, the ELF binary must have an embedded signature, that is created by the '''sgx_sign''' tool, using signing keys provided by whomever is going to attest to trustworthiness of the enclave binary. The signer can be whomever authors the enclave, but in most cases, can also be an arbitrary third party.
== Benefit to Fedora ==
As a general purpose infrastructure technology, SGX can be applied to / used by a wide variety of scenarios / applications.


The critical point is that whomever is going to later verify the SGX enclave must be willing to trust whomever signed it. Such a trust relationship decision is dependent on the usage context of the enclave in question.
In the context of this change proposal, '''no application''' usage is intended to be introduced. This proposal is focused on infrastructure enablement.


==== Enclave usage classes ====
A followup change proposal will be made to introduce Intel TDX confidential virtual machines, which is anticipated to be the first end user facing usage of SGX technology. Attestation is the means by which a guest VM owner, can prove that their VM machine is running in confidential mode on genuine Intel hardware, as opposed to being in a "blue pill" environment. All currently shipping Intel CPUs which support TDX build attestation on top of SGX with OSS enclave code (the '''tdqe''' enclave), as opposed to embedding attestation in proprietary firmware.


Broadly speaking there are considered to be two classes of enclave:
NB, the '''tdqe'' enclave will be included in this proposal, but will remain unused until the followup TDX change proposal introduces the QEMU changes that depend on it.


# '''Architectural enclaves''' - these are a small set of common enclaves, created and signed by Intel, which ship as standard in any SGX deployment. They provide a hardware enablement role, bootstrapping the cryptographic root of trust on a platform.
== Scope ==
# '''Application enclaves''' - this is an arbitrary set of custom enclaves, written by independent software vendors / providers to perform application specific tasks. They are signed by whichever party is to be relied upon to validate trustworthiness of the enclave code.
=== Proposal owners===


Both architectural and application enclaves are built and deployed in the same way, using the same runtime libraries. Their difference is largely where they sit in the chain of trust, with architectural enclaves being at the base, below all applications.
Add at least the following source packages to Fedora:


Given their use in bootstrapping the SGX platform chain of trust, the architectural enclaves would normally be considered to be serving the same role as platform firmware would. They establish an association with the physical platform hardware, which all other SGX uses relies upon.
* '''CppMicroServices''' - a C++ runtime library for building microservices daemons
* '''sgx-srpm-macros''' - define some common macros for where SGX content will live in the filesystem tree
* '''linux-sgx''' - provide the SGX platform development headers & libraries, runtime libraries, and supporting daemons


Application enclaves, meanwhile are clearly just userspace code, with a special protected execution context. They are created by ordinary application developers as needed to handle some specialized application tasks and have no relationship to platform firmware.
Assuming that pre-built, signed SGX architectural enclaves will be shipped, the following additional package is proposed:


==== Enclave execution context ====
* '''linux-sgx-enclavesXXX''' - one (or more) packages containing architectural enclaves


All enclaves run in ''ring 3'', which is traditionally the realm of userspace. The execution context, while using a region of memory allocated by & accounted to the loading application, is securely isolated from everything else on the system including the loading application. The enclaves' roles can look like either firmware, or kernel or userspace roles depending on the context on what the enclave is doing.  
Not every architectural enclave binary is re-issued on every release. New signed builds are only made available when there is a CVE fix, or a feature enhancement. At time of writing the SGX 2.25 release included new builds for every enclave. A future 2.26 release may only update a subset. Hence at some points in time, Fedora may have more than one '''linux-sgx-encalvesXXX''' package present.


Running in ''ring 3'' limits their ability to run privileged instructions, but the normal rules of access for rings do not apply. Nothing in ''ring 0''  ('''firmware''' / '''kernel'''), ''ring 3'' ('''userspace'''), or even ''psuedo-ring -1'' ('''SMM''') can interact with the enclaves, except via their published API entrypoints which are called via the EENTER instructions using a separate stack. IOW, the traditional view of hardware execution contexts being a simple stack of rings has been rather distorted by the capabilities of SGX, which exist as a distinct context off to the side of the rings model.
If it is required to perform a reproducible build of the above architectural enclave binaries, to prove the binaries match the claimed sources, the following toolchain packages are needed:


=== The architectural enclaves ===
* '''sgx-compat-gccXXX''' - one (or more) '''specific''' GCC versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''sgx-compat-binutilsXXX''' - one (or more) '''specific''' binutils versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''sgx-compat-nasmXXX''' - one (or more) '''specific''' NASM versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''sgx-compat-glibc-headersXXX''' - one (or more) '''specific''' GCC versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''sgx-compat-kernel-headersXXX''' - one (or more) '''specific''' GCC versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.


There are 8 so called "architectural enclaves" built, signed and distributed by Intel.
At time of writing, with the SGX 2.25 release, the versions required to perform a fully reproducible build will be


* '''libsgx_le.signed.so''' - "Launch Enclave (le)"
* '''gcc''': 8.5.0, 9.5.0
** This was historically used to control what enclaves are able to be launched on a platform. Modern systems allow loading on any enclave no matter who signed it.
* '''binutils''': 2.38, 2.40
** This enclave '''requires''' an Intel signature
* '''nasm''': 2.16.01
** ''This architectural enclave is considered obsolete & is NOT intended to be shipped in Fedora.''
* '''glibc''': 2.38
* '''kernel''': 5.17


* '''libsgx_qe.signed.so''' - "Quoting Enclave (qe)"
Note, while Fedora already ships builds fo gcc, binutils, etc, performing reproducible builds requires specific versions, configured with particular choice of build options. Hence the intention to ship parallel packages, which are '''exclusively''' intended for use in reproducing enclaves. There will be no support for using these toolchain packages for any other situation in Fedora.
** This was historically used in attestation of enclaves, using the Enhanced Privacy ID (EPID) scheme, which is replaced by RSA in SGX DCAP (Data Center Attestation Primitives) software.
** This enclave '''requires''' an Intel signature
** ''This architectural enclave is considered obsolete & is NOT intended to be shipped in Fedora.''


* '''libsgx_pve.signed.so''' - "ProVisioning Enclave (pve)"
Note: the fact that two versions are needed for binutils and GCC, is an upstream [https://github.com/intel/linux-sgx/issues/1045 an upstream bug]
** This was also historically used in attestation of enclaves, using the Enhanced Privacy ID (EPID) scheme, which is replaced by RSA in SGX DCAP (Data Center Attestation Primitives) software.  
** This enclave '''requires''' an Intel signature
** ''This architectural enclave is considered obsolete & is NOT intended to be shipped in Fedora.''


* '''libsgx_pce.signed.so''' - "Platform Certification Enclave (pce)"
Note: the enclaves are supposed to be built exclusively against the SGX SDK which provides its own C runtime headers. Thus the fact that it has a requirement for header files from glibc & kernel is also [https://github.com/intel/linux-sgx/pull/1062 likely another upstream bug].
** This certifies all other enclaves running on the system. The identity of the Intel's signing key, together with secrets inside the CPU set in fuses at time of manufacture, to derive a key that is known only to the enclave and Intel. This is needed in order to obtain a certificate from Intel that proves the hardware is genuine.
** This enclave '''requires''' an Intel signature
The full set of proposed source RPMs and their corresponding binary RPM outputs can be see from Copr:


* '''libsgx_id_enclave.signed.so''' - "Identification Enclave (ide)"
https://copr.fedorainfracloud.org/coprs/berrange/sgx-ng/monitor/
** This is used to extract some identification information about the processor that is needed to acquire a platform /processor certificate after deploying or re-provisioning a platform.
** This enclave '''recommends''' an Intel signature


* '''libsgx_qe3.signed.so''' - "Quoting Enclave (qe3)"
Note, the pre-built, signed architectural enclaves are not always re-created on each SGX release.
** This is used in attestation of enclaves, using the newer RSA scheme in SGX DCAP
** This enclave '''recommends''' an Intel signature


* '''libsgx_tdqe3.signed.so''' - "Trust Domain Quoting Enclave (tdqe)"
=== Other developers===
** This is used in attestation of TDX virtual machines, using the newer RSA scheme in SGX DCAP
** This enclave '''recommends''' an Intel signature


* '''libsgx_qve.signed.so''' - "Quote Verification Enclave (qve)"
It is not anticipated that other package maintainers need do anything to support introduction of SGX. The kernel is already built with the SGX feature enabled.
** This is used to provide a way to verify attestation reports on an otherwise untrusted machine using SGX DCAP.  
** This enclave '''recommends''' an Intel signature
** The pre-built binary for this enclave includes a build of openssl that potentially contains forbidden crypto & is NOT intended to be shipped in Fedora unless the crypto can be approved or eliminated


Out of the four architectural enclaves that '''require''' an Intel signature, only one is intended to be shipped in Fedora.
systemd ships with udev rules to enable creation of some of the required device nodes. The SGX package contains addon udev rules for remaining device nodes. The latter may be contributed upstream to systemd in future, and would flow back into Fedora in a normal systemd update.


While the remaining four architectural enclaves can theoretically have a non-Intel signature, if this is done it will increase friction of usage. The identity of the signer of enclaves becomes embedded in quotes, and is checked by the user (or software) validating the quote which is unlikely to be expecting a non-Intel signer for these architectural enclaves.
===Release engineering===


Thus it is considered that all shipped architectural enclaves need to have the Intel signature.
N/A - does not impact deliverables for releng


Users who wish to change & rebuild architectural enclaves from source can sign the result with their own keys, for all except the '''libsgx_pce.signed.so''' which unavoidably requires an Intel signature in order to derive the correct shared keys. If doing this, they will have to ensure that applications they use for verification are able to support use of non-Intel signers.
===Policies and guidelines===
 
=== Enclave licensing ===
 
All the code provided by the
 
The Fedora licensing guidelines describe the requirements for a pre-built binary to quality for the [https://docs.fedoraproject.org/en-US/legal/license-approval/#_licenses_allowed_for_firmware firmware exception] allowing source to be omitted.
 
 
* ''The files must be non-executable within the Fedora Linux context (note: this means that the files cannot run on their own, not that they are just chmod -x)''.
** ✅ Enclaves are distributed as ELF executable files, but this is just a convenient binary format. The ld-linux.so dynamic loader is unable to execute enclave binaries, and indeed a [https://github.com/intel/linux-sgx/pull/1061 RFE upstream] is proposing to eliminate the ''''.interp'''' ELF section contents to make this clear. They must be loaded using specialized SGX supporting code that sets up the enclave execution context on the processor.
 
* ''The files must not be libraries, within the Fedora Linux context''.
** ✅ While the enclave code is distributed as an ELF executable file, with a '''.so''' file extension, these are not shared libraries in the sense that Fedora knows them. The '''ld''' tool cannot link a binary to an enclave '''.so''' file. They must be loaded using specialized SGX supporting code that sets up the enclave execution context on the processor.
 
* ''The files must be standalone, not embedded in executable or library code (within the Fedora Linux context)''
** ✅ Every enclave is distributed as a standalone ELF executable file
 
* ''The files must be necessary for the functionality of open source code being included in Fedora Linux or to enable Fedora Linux to boot on a specific device, where no other reliable and supported mechanisms exist''
** ✅ Architectural enclaves. Users of SGX need to verify the chain of trust ultimately links back to a certificate issued by Intel, that attesting that the physical CPU was manufactured by them. The need for Intel to be the root signer is a physical hardware constraint, since only they are able to validate keys that are derived from secrets embedded in the CPU at time of manufacture.
** ❌ Application enclaves. It is valid for enclaves that provide application specific code to be signed by an arbitrary party. Whether a user trusts the signing party is an arbitrary decision they make. Not all uses of SGX will be viable when Fedora acts as a signer, and where not, such enclaves should be omitted from Fedora.
 
== Feedback ==
 
 
== Benefit to Fedora ==
As a general purpose infrastructure technology, SGX can be applied to / used by a wide variety of scenarios / applications.
 
The primary goal in introducing SGX into Fedora, however, is to support the Fedora KVM virtualization stack when it introduces confidential virtual machines running with Intel TDX. The TDX attestation implementation in currently integrated with Intel CPUs is built on the SGX technology. Attestation is the means by which a guest VM owner, can prove that their VM machine is running in confidential mode on genuine Intel hardware, as opposed to being in "blue pill" environment.
 
== Scope ==
=== Proposal owners===
 
Add the following packages to Fedora
 
* '''CppMicroServices''' - a C++ runtime library for building microservices daemons
* '''sgx-srpm-macros''' - define some common macros for where SGX content will live in the filesystem tree
* '''sgx-compat-gccXXX''' - one (or more) '''specific''' GCC versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''sgx-compat-binutilsXXX''' - one (or more) '''specific''' binutils versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''sgx-compat-nasmXXX''' - one (or more) '''specific''' NASM versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''sgx-compat-glibc-headersXXX''' - one (or more) '''specific''' GCC versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''sgx-compat-kernel-headersXXX''' - one (or more) '''specific''' GCC versions, built with ''targetted configure arguments'', to match the GCC configuration required for enclave reproducible build.
* '''linux-sgx-enclavesXXX''' - one (or more) packages for performing a reproducible build of architectural enclaves
* '''linux-sgx''' - provide the SGX platform development headers & libraries, runtime libraries, and supporting daemons


The pre-built, signed architectural enclaves are not always re-created on each SGX release. New binaries are only issued if there was a required feature change, or a security fix required. Thus the package for the reproducible enclave builds is separated out from the general SGX package. Furthermore, to perform a reproducible build of the full set of enclaves, requires potentially more than one '''linux-sgx-enclavesXXX''' package. For example SGX 2.23 and 2.24 releases only updated the ''''qve'''' enclave, all other enclaves remained on version 2.22. This in turn creates the requirement for multiple '''GCC'''/'''binutils'''/'''nasm''' versions, at some points.  The current newest 2.25 release has updated all enclaves, so initially only a single ''''linux-sgx-enclaves-2_25'''' is expected to be required.
A decision is needed around the handling of the pre-built, signed SGX enclaves. There are some relevant parallels in the guidelines for firmware as well as current practice wrt shim and CPU microcode.


NB, [https://github.com/intel/linux-sgx/issues/1045 an upstream bug] has resulted in different parts of the enclave code being built with different GCC/binutils versions. This should be resolved in future releases:
Normal Fedora practice requires building everything from source. There is a general exception to this for firmware blobs, which don't need to be built from source and don't need to be under an OSS compliant license as long as the binaries can be freely distributed. Hardware firmware binaries almost always include a digital signature from the vendor, to ensure that only trusted firmware is loaded onto the device. Thus even if source code is available for a given firmware binary, Fedora would still need to be shipping it as a pre-built vendor supplied signed binary.


NB, The requirement for glibc and kernel header packages is [https://github.com/intel/linux-sgx/pull/1062 likely another upstream bug]. The enclave code should be build exclusively against the SGX runtime, which is a completely custom C library. This is being investigated upstream.
Another example of shipping pre-built vendor binaries would be CPU microcode. This could be considered firmware for the core CPU, and again are vendor supplied, signed binaries, with no source code available.


=== Other developers===
A slightly different reference point is in the handling of ''shim'', where Fedora builds a binary from known good source, possibly with local pathes, and sends it off for signing by Microsoft, packaging the binary that is sent back. Microsoft has various requirements before it will permit signing a vendor's shim.


The kernel functionality required for SGX is already present in Fedora kernel packages, so no work by other maintainers is anticipated.
Consider if Fedora built a pristine shim release, with a designated toolchain version, using a reproducible build process. There would be no need to send the binary to Microsoft for signing, as the binary built in Fedora would match a standard pre-published signed binary from that shim release + toolchain. This is the situation SGX architectural enclaves are in. The main difference between the shim & SGX scenarios is that Fedora has the ability to add arbitrary local patches to shim & build with arbitrary toolchain versions of its choosing. This would not be possible when following a vendor specified reproducible build process.


===Release engineering===
In the shim case, it is often (but not always) possible for users to enroll their own certificates in UEFI to bless shim signatures from vendors other than Microsoft. IOW, a user often has the ability to build & customize shim themselves signing with their own keys. In the case of SGX enclaves, this is possible for 7, out of the 8 architectural enclaves, however the most fundamental enclave (''pce'') must always have an Intel signature to bootstrap the hardware root of trust.
 
N/A - does not impact deliverables for releng


===Policies and guidelines===
'''In summary''', the architectural enclaves meet the definition of firmware documented in the Fedora licensing guidelines:


Define policy requirements wrt shipping of pre-built SGX architectural enclaves (AEs) with cryptographic signatures from Intel.
* https://docs.fedoraproject.org/en-US/legal/license-approval/#_technical_firmware_requirements


Two options
Thus it should be permissible to ship the pre-built, signed binaries with no policy changes.


* Declare SGX architectural enclaves can be considered firmware, and thus exempt from build-from-source policy requirements.  
The shim model also gives impetus to the idea of it being acceptable to ship pre-built binaries, '''provided''' a reproducible build process can prove the binaries match the source under Fedora approved licenses.


* Define a policy explicitly for SGX enclaves, that permits shipping pre-built signed binaries for architectural enclaves only, '''if-and-only-if''', a reproducible build can validate the binaries.


NB, in all options, the proposed policy is '''ONLY''' intended to application to architectural enclave shipped by Intel. Any application that creates its own SGX enclaves would NOT be covered by this. ie this is '''NOT''' proposing to allow arbitrary 3rd party binary enclaves providing application code in Fedora, only the low level hardware enablement/bootstrapping enclave.
Ultimately it is suggested to consider a hybrid between the two views. Treat the SGX enclaves as firmware, but none the less require a reproducible build process, to prove the binaries correspond to the claim OSS code.


Application enclaves are considered out of scope for this feature proposal, but if required in the future, it is suggested that a Fedora signing key would be required for any application enclaves.
It is further suggested that this not be limited to SGX enclaves, instead expand the existing firmware exception guidance, to require a reproducible build is performed if the pre-built signed firmware binaries have complete & corresponding soruce available with a supported reproducible build process.


The distinction between architectural enclaves (signed by Intel to bootstrap trust) and application enclaves (signed by Fedora, with trust chained from the AEs), is considered conceptually similar to the distinction between shim (signed by Microsoft to bootstrap trust), vs kernel (signed by Fedora, with trust chained from shim).
The intent is to prove that the signed vendor firmware actually matches the published code whenever practical. This would apply to all SGX architectural enclaves, but also potentially to other distributed firmware with available source.


===Trademark approval===
===Trademark approval===
Line 224: Line 171:
Do you require 'QA Blueprint' support? N
Do you require 'QA Blueprint' support? N


The proposed new packages are available for testing via Copr:
The proposed new packages are available for testing via Copr, until such time as they are reviewed & built in Fedora koji:
 
* https://copr.fedorainfracloud.org/coprs/berrange/sgx-ng/


  https://copr.fedorainfracloud.org/coprs/berrange/sgx-ng/
These should work on any Intel Xeon class platform


== How To Test ==
== How To Test ==
Line 235: Line 184:
== User Experience ==
== User Experience ==


Initially minimal user experience impact, since on its own it doesn't deliver noticable end user features. Only once a followup proposal for integrating Intel TDX into KVM is done will the user experience changes.
Initially minimal user experience impact, since on its own it doesn't deliver noticeable end user features, as it is not believed that any existing applications in Fedora are able to leverage SGX.
 
The initial user benefit will be that users can bootstrap trust in SGX on their Fedora host. This will facilitate users in deploying 3rd party applications of their choosing that utilize SGX.
 
At a later time, when support for Intel TDX is integrated into KVM and QEMU, the immediate Fedora user benefit will significantly expand.


== Dependencies ==
== Dependencies ==


No existing packages will have a dependency on this change initially. In future, some deployments of QEMU may change to pull in certain SGX packages, to support Intel TDX.
The primary functional dependency for use of SGX is kernel support, which has existed in Fedora for some time. See "CONFIG_X86_SGX=y" in the kconfig files.


Kernel support for SGX already exists in Fedora:
This change proposal does not rely on work by another other package maintainers, but new packages will require reviewer attention.
 
  $ git grep CONFIG_X86_SGX=y
  kernel-x86_64-debug-fedora.config:CONFIG_X86_SGX=y
  kernel-x86_64-debug-rhel.config:CONFIG_X86_SGX=y
  kernel-x86_64-fedora.config:CONFIG_X86_SGX=y
  kernel-x86_64-rhel.config:CONFIG_X86_SGX=y
  kernel-x86_64-rt-debug-rhel.config:CONFIG_X86_SGX=y
  kernel-x86_64-rt-rhel.config:CONFIG_X86_SGX=y


== Contingency Plan ==
== Contingency Plan ==


<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->


* Contingency mechanism: The new packages have no ill effects on existing Fedora usage. Any outstanding work can be postponed to a later release if required.
* Contingency deadline: Beta free
* Blocks release? No


== Documentation ==
== Documentation ==
<!-- Is there upstream documentation on this change, or notes you have written yourself?  Link to that material here so other interested developers can get involved. -->


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
Documentation will be provided to describe:
N/A (not a System Wide Change)
 
* Configuring hardware UEFI to enable use of SGX
* Obtaining PCK certificates for a host from the Intel trusted services API
 
A subsequent change proposal will cover usage of SGX with TDX confidential virtual machines.


== Release Notes ==
== Release Notes ==

Latest revision as of 18:31, 30 October 2024

Intel SGX Software Stack

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

The Intel SGX technology enables creation of execution enclaves, whose memory is encrypted and thus protected from all other code running on the machine, including SMM, firmware, kernel and userspace. This proposal is to introduce the SGX host software stack to Fedora, to enable applications and features which have a dependency on SGX technology.

Owner

Current status

  • Targeted release: Fedora Linux 42
  • Last updated: 2024-10-30
  • [Announced]
  • [<will be assigned by the Wrangler> Discussion thread]
  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

The Intel SGX technology enables creation of execution enclaves, whose memory is encrypted and thus protected from all other code running on the machine, including SMM, firmware, kernel and userspace. While it has many potential use cases, this proposal is focused around the infrastructure needed to enable support for attestation of TDX confidential virtual machines.

The SGX software stack compromises a number of components

  • Support for developing new enclaves
    • Header files for the enclave code (a minimalist C library, C++ library, crypto and some other misc libraries)
    • Static library archives for linking into the enclave binaries.
    • Build helper tools (for signing enclaves, generating code enclave API entrypoints)
  • Support for developing applications that use enclaves, eg to be able to load and communicate with enclaves.
    • Header files for platform code
    • Dynamic libraries for platform code
  • Support for deploying applications that use enclaves
    • Enclave service daemon - assists unprivileged applications in loading enclaves
    • Registration tools - assists platform administrator in acquiring certificates to identity the platform
    • Quote generation daemon - assists QEMU in acquiring signed attestation reports for TDX VMs.

The code and binaries related to the host OS platform components will be installed in the normal filesystem locations common to all Fedora packages. e.g. libraries in /usr/lib64, headers in /usr/include and binaries in /usr/bin.

For the purposes of packaging, the enclaves will be treated as cross-compilation target. While the compiler build architecture target is x86_64, the runtime has custom C / C++ libraries that must be used, and requires a separate code loader. Enclaves cannot be directly linked to applications, they are strictly independent libraries.

With this in mind, all enclave related headers and libraries are proposed to be installed at /usr/x86_64-intel-sgx, specifically under lib64 and include sub-directories. There are no binaries for enclaves, only libraries, so no bin dir is required.

The generated binary packages will generally all have an 'sgx-' prefix to their name, with enclave related packages having an more specialized 'sgx-enclave-' name prefix.

Feedback

  • Objection: The SGX technology can be used as a way to implement DRM. Notable example has been BluRay playback.. Answer: As with many technologies, it is possible to use SGX in ways that are both positive and negative, wrt the owner / users of a machine. Use of SGX for DRM in BluRay playback is hostile to the owner/user of a machine. This change is NOT proposing to introduce / support any such usage/applications in Fedora. The fact that bad uses of SGX exist outside of Fedora, must not block the use of SGX in in Fedora for scenarios where it can offer features that benefit Fedora's users.
  • Objection: The SGX enclave code is not open source, because it requires a vendor signature on output binaries. Answer: 100% of the SGX code is made available under a variety open source licenses (Apache, BSD, MIT, GPL & more). One of the architectural enclaves, pce, requires an Intel signature because it is used to establish the root of trust with the hardware. In their role of bootstrapping use of SGX hardware, the architectural enclaves are a type of firmware, and it is normal for firmware to have a vendor signature. In contrast to almost all firmware which is proprietary, the architectural enclave code is all under Fedora approved OSS licenses. All of the architectural enclaves, can be rebuilt from source by a user, and signed with a user specified key if desired. If doing so, the limitation is that if the pce enclave is not signed by an Intel key, it will not be possible to obtain a certificate for the host from Intel's public PCS API service, and thus the integrity of SGX signed quotes cannot be validated back to the Intel root CA certificate.

Benefit to Fedora

As a general purpose infrastructure technology, SGX can be applied to / used by a wide variety of scenarios / applications.

In the context of this change proposal, no application usage is intended to be introduced. This proposal is focused on infrastructure enablement.

A followup change proposal will be made to introduce Intel TDX confidential virtual machines, which is anticipated to be the first end user facing usage of SGX technology. Attestation is the means by which a guest VM owner, can prove that their VM machine is running in confidential mode on genuine Intel hardware, as opposed to being in a "blue pill" environment. All currently shipping Intel CPUs which support TDX build attestation on top of SGX with OSS enclave code (the tdqe enclave), as opposed to embedding attestation in proprietary firmware.

NB, the 'tdqe enclave will be included in this proposal, but will remain unused until the followup TDX change proposal introduces the QEMU changes that depend on it.

Scope

Proposal owners

Add at least the following source packages to Fedora:

  • CppMicroServices - a C++ runtime library for building microservices daemons
  • sgx-srpm-macros - define some common macros for where SGX content will live in the filesystem tree
  • linux-sgx - provide the SGX platform development headers & libraries, runtime libraries, and supporting daemons

Assuming that pre-built, signed SGX architectural enclaves will be shipped, the following additional package is proposed:

  • linux-sgx-enclavesXXX - one (or more) packages containing architectural enclaves

Not every architectural enclave binary is re-issued on every release. New signed builds are only made available when there is a CVE fix, or a feature enhancement. At time of writing the SGX 2.25 release included new builds for every enclave. A future 2.26 release may only update a subset. Hence at some points in time, Fedora may have more than one linux-sgx-encalvesXXX package present.

If it is required to perform a reproducible build of the above architectural enclave binaries, to prove the binaries match the claimed sources, the following toolchain packages are needed:

  • sgx-compat-gccXXX - one (or more) specific GCC versions, built with targetted configure arguments, to match the GCC configuration required for enclave reproducible build.
  • sgx-compat-binutilsXXX - one (or more) specific binutils versions, built with targetted configure arguments, to match the GCC configuration required for enclave reproducible build.
  • sgx-compat-nasmXXX - one (or more) specific NASM versions, built with targetted configure arguments, to match the GCC configuration required for enclave reproducible build.
  • sgx-compat-glibc-headersXXX - one (or more) specific GCC versions, built with targetted configure arguments, to match the GCC configuration required for enclave reproducible build.
  • sgx-compat-kernel-headersXXX - one (or more) specific GCC versions, built with targetted configure arguments, to match the GCC configuration required for enclave reproducible build.

At time of writing, with the SGX 2.25 release, the versions required to perform a fully reproducible build will be

  • gcc: 8.5.0, 9.5.0
  • binutils: 2.38, 2.40
  • nasm: 2.16.01
  • glibc: 2.38
  • kernel: 5.17

Note, while Fedora already ships builds fo gcc, binutils, etc, performing reproducible builds requires specific versions, configured with particular choice of build options. Hence the intention to ship parallel packages, which are exclusively intended for use in reproducing enclaves. There will be no support for using these toolchain packages for any other situation in Fedora.

Note: the fact that two versions are needed for binutils and GCC, is an upstream an upstream bug

Note: the enclaves are supposed to be built exclusively against the SGX SDK which provides its own C runtime headers. Thus the fact that it has a requirement for header files from glibc & kernel is also likely another upstream bug.

The full set of proposed source RPMs and their corresponding binary RPM outputs can be see from Copr:

https://copr.fedorainfracloud.org/coprs/berrange/sgx-ng/monitor/

Note, the pre-built, signed architectural enclaves are not always re-created on each SGX release.

Other developers

It is not anticipated that other package maintainers need do anything to support introduction of SGX. The kernel is already built with the SGX feature enabled.

systemd ships with udev rules to enable creation of some of the required device nodes. The SGX package contains addon udev rules for remaining device nodes. The latter may be contributed upstream to systemd in future, and would flow back into Fedora in a normal systemd update.

Release engineering

N/A - does not impact deliverables for releng

Policies and guidelines

A decision is needed around the handling of the pre-built, signed SGX enclaves. There are some relevant parallels in the guidelines for firmware as well as current practice wrt shim and CPU microcode.

Normal Fedora practice requires building everything from source. There is a general exception to this for firmware blobs, which don't need to be built from source and don't need to be under an OSS compliant license as long as the binaries can be freely distributed. Hardware firmware binaries almost always include a digital signature from the vendor, to ensure that only trusted firmware is loaded onto the device. Thus even if source code is available for a given firmware binary, Fedora would still need to be shipping it as a pre-built vendor supplied signed binary.

Another example of shipping pre-built vendor binaries would be CPU microcode. This could be considered firmware for the core CPU, and again are vendor supplied, signed binaries, with no source code available.

A slightly different reference point is in the handling of shim, where Fedora builds a binary from known good source, possibly with local pathes, and sends it off for signing by Microsoft, packaging the binary that is sent back. Microsoft has various requirements before it will permit signing a vendor's shim.

Consider if Fedora built a pristine shim release, with a designated toolchain version, using a reproducible build process. There would be no need to send the binary to Microsoft for signing, as the binary built in Fedora would match a standard pre-published signed binary from that shim release + toolchain. This is the situation SGX architectural enclaves are in. The main difference between the shim & SGX scenarios is that Fedora has the ability to add arbitrary local patches to shim & build with arbitrary toolchain versions of its choosing. This would not be possible when following a vendor specified reproducible build process.

In the shim case, it is often (but not always) possible for users to enroll their own certificates in UEFI to bless shim signatures from vendors other than Microsoft. IOW, a user often has the ability to build & customize shim themselves signing with their own keys. In the case of SGX enclaves, this is possible for 7, out of the 8 architectural enclaves, however the most fundamental enclave (pce) must always have an Intel signature to bootstrap the hardware root of trust.

In summary, the architectural enclaves meet the definition of firmware documented in the Fedora licensing guidelines:

Thus it should be permissible to ship the pre-built, signed binaries with no policy changes.

The shim model also gives impetus to the idea of it being acceptable to ship pre-built binaries, provided a reproducible build process can prove the binaries match the source under Fedora approved licenses.


Ultimately it is suggested to consider a hybrid between the two views. Treat the SGX enclaves as firmware, but none the less require a reproducible build process, to prove the binaries correspond to the claim OSS code.

It is further suggested that this not be limited to SGX enclaves, instead expand the existing firmware exception guidance, to require a reproducible build is performed if the pre-built signed firmware binaries have complete & corresponding soruce available with a supported reproducible build process.

The intent is to prove that the signed vendor firmware actually matches the published code whenever practical. This would apply to all SGX architectural enclaves, but also potentially to other distributed firmware with available source.

Trademark approval

N/A

Alignment with the Fedora Strategy

This aligns with

  • "Reaching the world". Including SGX will make the Fedora support for hosting Intel TDX confidential virtual machines feature complete, by enabling attestation by the guest owner
  • "Innovation & leadership in technology". SGX is a general purpose infrastructure technology which enables application developers to build systems to securely run sensitive workloads.

Upgrade/compatibility impact

This is a new package set which should not have any upgrade impact, as it will not initially be a dependency of other software. In future it may be pulled in automatically as a dependency in certain KVM deployment scenarios. Even when installed, using anything related to SGX first requires host firmware changes to enable use of the technology. The systemd services provided have their unit files conditionalized on the existence of /dev/sgx_enclave device nodes.

Early Testing (Optional)

Do you require 'QA Blueprint' support? N

The proposed new packages are available for testing via Copr, until such time as they are reviewed & built in Fedora koji:

These should work on any Intel Xeon class platform

How To Test

  • Document how to validate a single-socket system by obtaining PCK certificates automatically
  • Document how to configure a multi-socket system to enable its registration with Intel services, and request a PCK certificate

User Experience

Initially minimal user experience impact, since on its own it doesn't deliver noticeable end user features, as it is not believed that any existing applications in Fedora are able to leverage SGX.

The initial user benefit will be that users can bootstrap trust in SGX on their Fedora host. This will facilitate users in deploying 3rd party applications of their choosing that utilize SGX.

At a later time, when support for Intel TDX is integrated into KVM and QEMU, the immediate Fedora user benefit will significantly expand.

Dependencies

The primary functional dependency for use of SGX is kernel support, which has existed in Fedora for some time. See "CONFIG_X86_SGX=y" in the kconfig files.

This change proposal does not rely on work by another other package maintainers, but new packages will require reviewer attention.

Contingency Plan

  • Contingency mechanism: The new packages have no ill effects on existing Fedora usage. Any outstanding work can be postponed to a later release if required.
  • Contingency deadline: Beta free
  • Blocks release? No

Documentation

Documentation will be provided to describe:

  • Configuring hardware UEFI to enable use of SGX
  • Obtaining PCK certificates for a host from the Intel trusted services API

A subsequent change proposal will cover usage of SGX with TDX confidential virtual machines.

Release Notes

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/IntelSGX&oldid=723618"