User:Mitr/NSS:DeveloperFriendliness

Documentation

Full API documentation in a single place

Developers should have a single, definitive place to find all relevant reference documentation.

Probably it would be ideal to have documentation integrated in the source code, extracted at build time (and installed with the nss-devel RPM); this would make it easiest to update/reference during development (however, having all documentation in a single place is more important than having it in the source code).

We should have everything documented, priority items:

• TLS (documentation mostly exists)
• Basic crypto (encryption/hashing/signatures) (covered in tech note 5 only)
• How to initialize NSS (no-DB, system-wide DB, other possibilities, when to use which one)
• How to use the PEM token / interoperate with OpenSSL conventions (might want to improve the API first)

Known existing documentation:

• https://developer.mozilla.org/en-US/docs/NSS top-level
• https://developer.mozilla.org/en-US/docs/NSS_reference contains structured documentation to a subset of functions
• https://developer.mozilla.org/en-US/docs/NSS_functions has a more complete list of functions, but mostly only links to MXR.
• Some header files contain (unformatted) documentation comments.
• https://developer.mozilla.org/en-US/docs/NSS/NSS_Tech_Notes contains various examples, notably https://developer.mozilla.org/en-US/docs/NSS/nss_tech_notes/nss_tech_note5 is the only(?) documentation of low-level cryptography facilities.

NSS bug for docbook use: #912360

Smallish API improvements

(Can't break binary compatibility - will typically have to introduce new functions.)

Fix individual warts in the existing API

• For ciphers with padding it is necessary to call PK11_DigestFinal() to get the last encrypted block.

NSS bug: #884178

python-nss bug: FIXME

Make it easier to integrate TLS user in UNIXy environments/applications

• Make PR_ImportTCPSocket() a fully supported API (for UNIX at least, for applications that have no ambition to be portable to native Windows API).

NSS bug: #924376

python-nss bug: not filed yet

• Add a "poll(2) flags" interface that would make it possible to integrate non-blocking NSS TLS sockets into an existing event loop infrastructure (currently it seems necessary to replace the existing event loop around one built on PR_Poll, which is too invasive for large applications)

NSS bug: #888581

python-nss bug: FIXME

• • Notes:
    • Bob: The big gotcha on non-blocking sockets is to handle on the fly prompts. We have to deal with the problem where the application throws up a dialog to, say, select a client auth cert, or decide to accept a failed cert. We could define the problem away (say that if you're application blocks in any SSL callback, you can't use non-blocking). Not an insurmountable problem, just one that perennially pops up.
    • Stef: gtls provides this for GLib based applications: In the GLib gtls backend we always handshake in another thread, even though we support doing all the bulk SSL reads/writes in the event loop in a 'pollable' fashion. We came to the realization that even with polling implemented that it's nearly impossible to do a TLS handshake in an event loop thread without big latency due to the following:
      • RSA operations on key sizes that are 'large' given the hardware introduce noticeable to unnacceptable latency.
      • PKCS#11 and other smart card operations are inherently blocking (there are a few exceptions).
    • Rich:
      • Some applications are not threaded and/or cannot use threads
      • SSL_ForceHandshake() is re-entrant (FIXME: ?)

Allow handling handshakes in non-blocking applications

Related to the above: allow setting a mode in which NSS will return an error instead of automatically doing a handshake, or add a callback when a handshake starts.

Contact: Dan Winship

NSS bug: #924381

python-nss bug: not filed yet

Minimize boilerplate

This is fairly general; e.g. avoid SECItem structures - they make it easy to pass data through layers, but at the top-level layer using them is more code than just passing a (void *, size_t) pair

NSS bug: #924390

python-nss bug: not filed yet

Make it easier to use NSS for basic cryptography

Add a set of "easy to use"/"lite" alternatives to pk11pub.h and related headers, to be at least as easy as PR_Connect()/PR_Read()/PR_Write() for TLS. At the very least, hide the existence of slots.

Together with avoiding SECItem, might involve primarily adding symmetric encryption to the cryptohi layer.

Would it make sense to make it also easy to import raw key material for an one-shot operation? Encourages handling raw key material directly, which is not too desirable.

NSS bug: #924396

python-nss bug: not filed yet

Make it really easy to initiate a TLS TCP connection

E.g. for clients, combine the DNS lookup with PR_Connect + SSL_SetURL etc to make the common case trivial.

NSS bug: #924401

python-nss bug: not filed yet

Minimize the amount of magic necessary for loading CA configuration (and other data) from files

E.g. hide the existence of the PEM token.

NSS bug: #924404

python-nss bug: not filed yet

Allow applications to enumerate cipher suites / mechanisms

(to avoid/minimize the need for https://bugzilla.redhat.com/show_bug.cgi?id=970727 )

NSS bug: #480174

python-nss bug: not filed yet

Defaults / System Policy Integration

Individual applications should not be forced or encouraged to individually enable/disable various mechanisms; this should primarily happen at an OS-wide policy level.

Setting the right developer/user expectations in the API is the primary challenge (when an algorithm is broken, do we keep the application running or do we keep the application secure by disabling the algorithm)?

Make it easy to use secure defaults

Already discussed as https://bugzilla.mozilla.org/show_bug.cgi?id=842307 .

Make it really easy to initialize a client for using the system-wide CA database and policy

Allow loading the default CA trust without manually managing the database or loading magic modules, or at least document it well.

Multiple library contexts

This is currently fairly vague and expected to be difficult - however there is definite demand to allow several independent uses of NSS within an application (even within a single thread), with various modules independently setting their configuration and policy.

This involves at least CA trust configuration, not sure about individual mechanisms as set by e.g. NSS_SetDomesticPolicy (though recommending application writers to use an "OS policy" everywhere might be a way to avoid this).

References:

• https://wiki.mozilla.org/NSS_Library_Init
• https://wiki.mozilla.org/NSS_Shared_DB_And_LINUX

Better smart-card login support

To get OpenSSL feature parity and allow using NSS in Fedora builds of Kerberos (and sssd?)

Support PKINIT certificate usage / trust

... when validating certificates, in addition to the existing TLS / email / code signing.

Fedora bug: #961113

NSS bug: #383601

python-nss bug: Not filed yet

Support validating certificates with other custom certificate usages

May be necessary for non-Kerberos smart card login mechanisms.

Get the NSS PEM module accepted upstream

Would help with NSS code acceptance in Kerberos.

Fedora bug: #804215

NSS bug: #402712

Alternative(?) DN<=>text mapping functions

As we implement smart card authentication in SSSD, we're going to want to be able to leverage printable versions of subject names which weren't generated by NSS, and which may include more attributes than NSS knows the names of, and which may refer to various attributes by different names (for example, "E" vs "emailAddress"), for example altSecurityIdentity attributes read from an Active Directory LDAP entry.

Being able to reliably derive the same string representation that the AD administrative tools did when populating that attribute (whether we accomplish that by being able to supply NSS with a custom NameToKind dictionary for CERT_AsciiToName (? or CERT_NameToAscii?), or by implementing an entire other DN-to-text conversion function) is going to be necessary for making that work.

PEM support enhancements

... for better OpenSSL compatibility:

Support reading CRLs

Fedora bug: #871670

Support encryption/decryption using AES and Camellia

Fedora bug: [1]

Performance

Try to improve performance - [Milan Bartoš' bachelor thesis] measures a significant performance disadvantage, especially for small data sizes.

Notes on crypto use in OpenStack

(Only vaguely related - split?)

General notes

• Universally stores keys, certificates, and the like, in files, or as text within configuration files.
• TLS is used through several implementations/wrappers, including at least the built-in ssl module, httplib2 and python-requests
• Most (all) TLS configurations include an option to run in "insecure" mode

(Uses of crypto within the test suite have been ignored)

OpenSSL library uses

Through PyOpenSSL:

• glance/glance/common/utils.py : OpenSSL.crypto.{load_private_key,load_certificate,sign,verify}, only to validate the key/certificate public/private is valid and they match
• tempest/tempest/common/glance_http.py : OpenSSL.SSL.Connection: does its own peer verification
• python-glanceclient/glanceclient/common/http.py : OpenSSL.SSL.Connection: does its own peer verification
• python-swiftclient/swiftclient/https_connection.py : OpenSSL.SSL.Context

OpenSSL command-line uses

There are several instances of managing a CA (using the OpenSSL ca directory structure), and using openssl(1) as a data filter.

... and the used subcommands:

• keystone/keystone/common/openssl.py : genrsa, req, ca
• keystone/keystone/common/cms.py : cms -verify, cms -sign
• python-keystoneclient/keystoneclient/common/cms.py : cms -verify, cms -sign
• keystone/examples/pki/gen_pki.sh ~= python-keystoneclient/examples/pki/gen_pki.sh : req, ca, x509, cms -sign (used to generate test data only?)
• trove/trove/guestagent/strategies/restore/base.py : enc -aes-256-cbc -salt -pass
• trove/trove/guestagent/strategies/backup/base.py : enc -aes-256-cbc -salt -pass
• python-novaclient/novaclient/crypto.py : rsautl -decrypt
• nova/doc/source/devref/rc.local : openssl(1) enc -d, in a shell script
• nova/nova/cloudpipe/bootscript.template : openssl(1) rand, dhparam, in a shell script
• nova/nova/CA/geninter.sh : openssl(1) req, ca -gencrl
• nova/nova/CA/genrootca.sh : openssl(1) req, ca -gencrl
• nova/nova/crypto.py : rsautl -{decrypt,encrypt}, ca -{revoke,gencrl}, genrsa, req, x509 -serial (also CA directory management)
• nova/nova/image/s3.py : enc -d -aes-128-cbc
• nova/nova/virt/xenapi/agent.py : aes-128-cbc

Related:

• tempest/tools/install_venv.py : installs openssl-devel on CentOS
• nova/nova/CA/openssl.cnf.tmpl : OpenSSL configuration file

python-crypto

• keystone/keystone/openstack/common/crypto/utils.py ~= trove/trove/openstack/common/crypto/utils.py ~= oslo-incubator/openstack/common/crypto/utils.py: Crypto.Hash.HMAC; CBC encryption; exposes Crypto.Hash.HMAC; with optional base64 en/decoding
• python-keystoneclient/keystoneclient/middleware/memcache_crypt.py : Crypto.Cipher.AES in CBC mode
• python-openstackclient/openstackclient/common/openstackkeyring.py : Crypto.Cipher.AES in CFB
• glance/glance/common/crypt.py : Crypto.Cipher.AES, Crypto.Random: AES-CBC
• heat/contrib/rackspace/heat/engine/plugins/cloud_server.py : Crypto.PublicKey.RSA: RSA.{importKey,generate}.{exportKey,publickey().exportKey()}
• heat/heat/common/crypt.py : Crypto.Cipher.AES: CFB, base64

python-eventlet

• swift/swift/common/bufferedhttp.py : eventlet.green.httplib.HTTPSConnection
• heat/heat/common/wsgi.py : eventlet.green.ssl.wrap_socket
• swift/swift/common/wsgi.py : eventlet.green.ssl.wrap_socket
• glance/glance/common/wsgi.py : eventlet.green.ssl.wrap_socket
• nova/nova/wsgi.py : eventlet.wrap_ssl

python-paramiko

... as a SSH client:

• cinder/cinder/utils.py
• cinder/cinder/volume/drivers/san/hp/hp_3par_common.py
• heat/contrib/rackspace/heat/engine/plugins/cloud_server.py
• ironic/ironic/common/utils.py
• nova/nova/virt/powervm/common.py
• tempest/tempest/common/ssh.py

Misc. other libraries

• keystone/keystone/common/utils.py : passlib.hash.sha512_crypt, passlib.hash.ldap_salted_sha1 (password hashing)
• heat/heat/common/crypt.py : Uses os.urandom to generate an IV
• openstack-chef/cookbooks/openssl/libraries/secure_password.rb : OpenSSL::Random::Random_bytes

Python standard library

(Ignoring very many hashlib uses, and many hmac uses.)

• nova/nova/virt/disk/api.py : crypt.crypt (MD5 or DES)
• cinder/cinder/volume/drivers/zadara.py : httplib.HTTPSConnection
• keystone/keystone/middleware/s3_token.py : httplib.HTTPSConnection
• neutron/neutron/plugins/bigswitch/plugin.py : httplib.HTTPSConnection
• neutron/neutron/plugins/nec/common/ofc_client.py : httplib.HTTPSConnection
• neutron/neutron/services/loadbalancer/drivers/radware/driver.py : httplib.HTTPSConnection
• python-swiftclient/swiftclient/client.py : httplib.HTTPSConnection
• glance/glance/common/client.py : httplib.HTTPSConnection, ssl.wrap_socket
• ceilometer/ceilometer/openstack/common/sslutils.py ~= heat/heat/openstack/common/sslutils.py~= keystone/keystone/openstack/common/sslutils.py ~= nova/nova/openstack/common/sslutils.py ~= oslo-incubator/openstack/common/sslutils.py ~= oslo.messaging/oslo/messaging/openstack/common/sslutils.py ~= trove/trove/openstack/common/sslutils.py : ssl.wrap_socket
• cinder/cinder/wsgi.py : ssl.wrap_socket
• neutron/neutron/wsgi.py : ssl.wrap_socket
• nova/nova/scheduler/filters/trusted_filter.py : ssl.wrap_socket
• python-ceilometerclient/ceilometerclient/common/http.py : ssl.wrap_socket
• python-heatclient/heatclient/common/http.py : ssl.wrap_socket

Custom crypto implementation

• keystone/keystone/openstack/common/crypto/utils.py ~= trove/trove/openstack/common/crypto/utils.py ~= oslo-incubator/openstack/common/crypto/utils.py : Implements HKDF (RFC 5869) based on Crypto.Hash.HMAC. Manually implements ANSI X.923 padding.
• python-keystoneclient/keystoneclient/middleware/memcache_crypt.py : Derives a key using HMAC+SHA384. Implements PKCS#7 padding.
• glance/glance/common/crypt.py : Implements a custom padding scheme.
• nova/nova/crypto.py : Implements conversion from SSH key format to PKCS#8 (base64+ASN.1)
• nova/nova/virt/xenapi/agent.py : Implements D-H key exchange.
• (nova/nova/virt/disk/api.py : Custom code editing /etc/shadow )