Using an HSM to protect your encryption and signing keys

/ Martijn Brinkers

Like any application that uses private keys, there is always the issue on how to securely store sensitive private key material. The CipherMail gateway stores all settings, including keys and certificates, in a database. One of the benefits of storing all data in a database is that it makes it simple to do backups, full clustering and fail-over.

Even though all sensitive data, like for example private keys, is encrypted with a configurable password, anyone with access to the database contents and the system password might be able to get access to the private keys. This is not specifically a problem of the CipherMail gateway. Any application that uses private keys, and which does not use specialized hardware to securely store the private key material, has the same problem. It is therefore important that access to the database is only allowed to authorized personnel and that system backups of the gateway are encrypted with a strong password.

To make sure that private keys can never be copied, even with full physical access, a Hardware Security Module (HSM) should be used. An HSM is basically a big smart card. It generates private keys directly on the device and stores the private keys on tamper proof hardware. An HSM also provides additional security functionality like for example a built-in secure random generator. For FIPS 140 level 2 and up, an HSM is required because FIPS 140-2 requires physical security mechanisms.

Using an HSM for encryption and digital signing of email can be important if you want to use qualified certificates. For example one of our clients issue their own trusted certificates from an external CA (EJBCA) connected to the gateway. If the gateway is instructed to digitally sign email, the gateway can automatically request an S/MIME signing certificate from the external CA. The private key for the signing certificate is generated on the attached HSM and the certificate signing request (CSR) is sent to the CA which will then automatically issue the certificate.

By using an HSM your security level can be drastically improved. The downside of an HSM is that an HSM can be expensive. You need at least two, preferably in a clustered setup, in case one breaks down. The CipherMail gateway can use an HSM for S/MIME and PGP keys (most other gateways do not support storing PGP keys on an HSM). The CipherMail gateway has been tested with Thales nCipher, Utimaco and Safenet HSMs.