Symmetric Encryption in Java

Categories: Java, Cryptography

(back to main post)

Introduction

A previous article presented the basic terminology and algorithms of symmetric encryption; this article discusses the related classes from the Java standard library, and from the alternative open-source BouncyCastle cryptography library.

General Comments on the Standard Java Cryptography APIs

The jdk has an extensive set of classes in the java.security and javax.crypto packages for performing encryption, decryption, signing, and other such operations. Sadly these APIs are a disaster. Due to an attempt to provide “pluggable” security (cryptography) libraries, the API involves a weird mismatch of completely type-unsafe methods and types. For absolutely no good reason it then adds classes of rare and extremely specific purpose, names that seem to be deliberately confusing (security through obscurity?) and strange inheritance structures. To round it off, the javadoc is often incomplete or ambiguous.

Fortunately there are some people out there who understand this maze and have posted good advice that can be found via appropriate searches. Sadly, there are also a lot of people out there who misunderstand this maze and have posted dangerously bad advice that is found via the same search terms. Beware in all directions!

Hopefully the information in this article is correct; if it is not then please let me know immediately and the article will be corrected.

Standard Java Cryptography APIs

Java cryptography classes are found in either java.security or javax.crypto.

The package java.security is mostly about asymmetric encryption (ie public/private key encryption). This is because the java environment itself uses asymmetric encryption for:

  • signing jarfiles (and thus being able to grant them specific permissions) - which also requires message digest (ie hashing) support
  • implementing SSL/TLS ie https; an https server identifies itself by providing a certificate in which is embedded a public key

The addition of symmetric encryption features was done later, and these are mostly in the javax.crypto package.

Some Standard Java Classes

The javadoc on a number of critical classes is - less than optimal, so here is some additional information on the most important classes. This list contains about 10% of the classes under java.security and 30% of the classes under javax.crypto; the rest are not relevant for symmetric encryption.

The entries are grouped in an order that the following sections will hopefully make clear.

All JDK classes relevant for symmetric encryption

  • javax.crypto.Cipher
  • javax.crypto.CipherInputStream
  • javax.crypto.CipherOutputStream
  • java.security.Key
  • javax.crypto.SecretKey
  • javax.crypto.interfaces.PBEKey
  • java.security.spec.KeySpec
  • javax.crypto.spec.SecretKeySpec
  • javax.crypto.spec.PBEKeySpec
  • javax.crypto.SecretKeyFactory
  • java.security.AlgorithmParameters
  • java.security.spec.AlgorithmParameterSpec
  • javax.crypto.spec.IvParameterSpec
  • java.security.AlgorithmParameterGenerator
  • javax.crypto.KeyGenerator
  • javax.crypto.Mac
  • javax.crypto.spec.PBEParameterSpec

Ciphers and Streams

Class javax.crypto.Cipher represents an object that is capable of turning plaintext into ciphertext and reverse. When an instance is created (via the getInstance static factory methods) it has an internal algorithm but does not have the “configuration” necessary to actually encrypt or decrypt. One of the many init methods must be called to provide it with the necessary configuration data. In the case of symmetric encryption, the data needed is:

  • a symmetric key (ie a SecretKey object) - which might be an already-prepared block of bytes, or data from which one can be generated by the Cipher (see PBEKey later)
  • usually an IvParameterSpec object to specify the Initialization Vector to use

A Cipher instance (as returned from a getInstance method) is actually simply a wrapper class that delegates to an object that does the real work - one created by a “pluggable security provider” implementation. The wrapper class is always the same type, provided by the JDK. And the wrapper enforces any restrictions on the encryption strength - the infamous maximum of 128-bit keys (see section below). There is no avoiding this limit when using the standard Cipher class.

The CipherInputStream and CipherOutputStream classes do show the benefit of unifying all types of encryption/decryption under a single Cipher interface; these classes transparently decrypt or encrypt data that flows through them and don’t care if the encryption is symmetric or asymmetric.

There are a number of bizarre API features in the Cipher class:

  • why are encryption and decryption unified, with an init parameter to set the “direction”? Surely separate Encipher and Decipher interfaces would have been clearer..
  • why is an instance created without already having the data necessary to work? The object returned from getInstance() is totally useless; why not have the getInstance methods take the parameters that init does?
  • actually, why have these bizarre init methods at all? The key and parameter objects passed to the Cipher have to have “algorithm” types that match the algorithm parameter passed to the Cipher.getInstance method or a runtime failure occurs. So why not have concrete Cipher subclasses and do away with the getInstance factory method, and the init method?

The reason for the above design flaws is the hidden fact that a Cipher instance is a standard wrapper that simply delegates to a real implementation supplied by a pluggable provider. However this is no consolation when having to use this API. In short, the Cipher API is so designed that it is impossible to use without documentation, and without carefully keeping the key-loading and cipher-creating parts of the code in sync. Failure to do this results in runtime failures, not compile-time ones.

Key types

Interface java.security.Key represents the magic data that a cipher needs to encrypt or decrypt data. Instances of type Key are usually created by a KeyFactory (asymmetric) or SecretKeyFactory (symmetric) from a KeySpec. For symmetric encryption, the resulting object is always a subtype of javax.crypto.SecretKey.

Interface javax.crypto.SecretKey is a subtype of Key that represents a symmetric key. For some algorithms (eg AES or DES), the key object is just a wrapper around an array of random bytes, and the getEncoded method simply returns those raw bytes.

Interface javax.crypto.interfaces.PBEKey is a subtype of SecretKey that wraps values from which a “real” symmetric encryption key can be derived: (password, salt, iteration). The type is actually seldom referenced directly in Java code, as they are generated by a SecretKeyFactory from a PBEKeySpec - and the return type of a SecretKeyFactory is SecretKey, not PBEKey. Ciphers that require a PBEKey will throw a runtime exception if passed a Key instance which is not a PBEKey. Such Ciphers knows how to internally derive a real key from the password and associated parameters encapsulated in the PBEKey object. Bizarrely, although instances of this type must be generated by a SecretKeyFactory (see later) from a KeySpec, this class also simultaneously extends KeySpec - because it does remember the parameters with which it was created. The generic SecretKey interface can’t do that because it is a more abstract type that doesn’t record how its internal key (byte-array) was actually created.

The API is somewhat weird: types are scattered across various packages, and while there is a specific interface for PBEKey there is none representing a SecretKey derived directly from a byte-array (eg “RawSecretKey”). Yes, a RawSecretKey interface would be redundant as it couldn’t offer any more methods than are present on its parent type but it would make the inheritance tree more symmetrical and comprehensible.

KeySpec types

Interface java.security.spec.KeySpec encapsulates “the values from which a key can be derived”. Generally, a KeySpec instance is fed into a matching KeyFactory instance resulting in a Key object. As an example from asymmetric encryption, a DSA key is derived from numbers named p, q, x and d which satisfy specific mathematical constraints.

Class javax.crypto.spec.SecretKeySpec represents a byte-array from which a secret-key can be derived via the trivial process of um - doing nothing, because the byte-array is the key. Because in this case the specification is identical to the key, the SecretKeySpec class also implements the SecretKey interface - unlike other keyspec classes such as PublicKeySpec, PrivateKeySpec and PBEKeySpec. While short-circuiting the normal type structures (making this spec class implement a key interface directly) makes sense from an implementation (and maybe performance) point of view, it certainly doesn’t assist understanding the cryptography classes when this type directly implements an unexpected parent interface just because the normal transformation (spec -> keyfactory -> key) happens to be an identity transformation in this case. Sigh. The fact that this class has a bad name - IdentityKeySpec would perhaps have been better - doesn’t help. Sigh again.

Class javax.crypto.spec.PBEKeySpec represents “Password Based Encryption” parameters from which a SecretKey can be derived - password, salt and iteration-count.

Note that the “keyspec” types don’t actually define the algorithm which should be used to transform the “specification” into a key. That is the role of a KeyFactory.

Key Factories

A key-factory converts a key-spec into a key. As an example from asymmetric encryption, an appropriate keyfactory can unpack a certificate into a public-key object.

From the naming patterns above, you would expect that java.security.KeyFactory is a base type representing the abstract concept of turning a KeySpec into a Key, right? And that SecretKeyFactory extends KeyFactory. Nah, far too obvious and consistent for the designers of the JDK crypto APIs. In fact, although the classes are indeed doing the same logical thing (keyspec -> key), they are not related at all. KeyFactory is the original class designed for dealing with asymmetric encryption as present since the early versions of Java (1.2?). SecretKeyFactory was added when symmetric-encryption support was added and fulfils the same role but is not related.

The implementation of both KeyFactory and SecretKeyFactory are similar to the Cipher class: a single common “wrapper” class that delegates to an actual implementation created by a “pluggable security provider”.

Actually, it is good that SecretKeyFactory does not subclass KeyFactory; there is no benefit to it (and KeyFactory has asymmetric-specific methods like generatePrivateKey). However it would have been good if the designers had also:

  1. not chosen class names that suggest a parent-type/subtype relation, and
  2. not taken exactly the opposite approach with Key/SecretKey and KeySpec/SecretKeySpec, where type/subtype relations do exist.

The reason for (2) is that Key/SecretKey and KeySpec/SecretKeySpec are data objects that are not implemented by a “pluggable security provider”, while Cipher and SecretKeyFactory implementations are. However IMO that is no justification for such naming chaos.

Obviously, specific KeyFactory implementations (algorithms) accept only specific KeySpec types, and return the corresponding Key types - which must then be fed into an appropriate Cipher type. A mismatch anywhere results in a runtime failure.

This is (IMHO) yet another API design failure; keyfactories are so tightly coupled to their input keyspec types that it makes no sense for there to be a common abstract type for doing this. The API has been seriously distorted to allow different algorithm implementations with various different configuration needs to be squeezed through a common “service provider” interface.

To convert a PBEKeySpec to a SecretKey, an appropriate KeyFactory needs to be used. Method SecretKeyFactory.generateSecret returns an instance of type SecretKey - but this can be downcast to PBEKey if (and only if) the input is a PBEKeySpec and the algorithm is a PBE algorithm. Yet another type-unsafe interface.

And oddly, because the transformation from SecretKeySpec to SecretKey is an identity translation (see comments in previous section), there is actually no need to use a SecretKeyFactory for this mapping; just use the SecretKeySpec directly - as it also implements SecretKey itself.

Algorithm Parameters

The java crypto APIs have to funnel data from the application to “security provider” implementations, so rather than have sane simple typesafe APIs, they use generic types and collections of those types. In particular, java.security.AlgorithmParameters is a collection of “stuff that some class in a security provider library might need to know”. All those “things a library might need to know” must be represented as classes subclassing java.security.spec.AlgorithmParameterSpec.

The most significant user is the Cipher class. Its init method can be passed an AlgorithmParameters object containing data that the underlying wrapper implementation needs. Sadly, this is completely type-unsafe; the API gives no hint as to what is needed (because what is needed depends upon the string passed to the getInstance method), and if the wrong objects are passed then an exception occurs at runtime.

Fortunately, there is only one algorithm parameter used by Cipher implementations for symmetric encryption: javax.crypto.spec.IvParameterSpec containing the InitialisationVector for any cipher algorithm that uses a “block chaining mode” that requires an initialisation vector (and you should always be using a block chaining mode).

  IvParameterSpec ivSpec = new IvParameterSpec(someBytes);
  cipher.init(opmode, key, ivSpec);

  // or this - although as Iv is the only thing you'll ever need to store in the params, it is somewhat pointless..
  //
  // AlgorithmParameters algoParams = AlgorithmParameters.getInstance("AES-CBC");
  // algoParams.init(ivSpec);
  // cipher.init(opmode, key, algoParams);

The article on symmetric encryption describes what an IV is, and how you might get one.

There is also a class (java.security.AlgorithmParameterGenerator) for generating algorithm parameters. Presumably it could be used to generate an IV when encrypting outgoing messages. However given that an IV is just a random sequence of bytes it is easier to just use SecureRandom.nextBytes for that.

Key Generators

Class javax.crypto.KeyGenerator supports creating new keys. As far as I am aware (and note that I am NOT a cryptographer), an AES key is just a random block of bytes and therefore it is just as valid to use SecureRandom.nextBytes to generate an AES key as to use javax.crypto.KeyGenerator. However for other key types this is not necessarily true, and using the KeyGenerator is required.

Some encryption algorithms do have specific ranges of numbers which are considered “weak keys”, ie which interact with the algorithm to generate output which is easier to decrypt than for most other keys; hopefully a KeyGenerator will know how to avoid returning such keys. However AES does not have any such ‘weak keys’.

Message Authentication Codes

A MAC algorithm (class javax.crypto.Mac) is a kind of secure-hash; hash algorithms such as MD5 or SHA1 do not have a “key”, but MAC algorithms do.

As noted in my article on symmetric encryption, block-mode algorithms such as AES will usually fail to decrypt data if it has been accidentally or deliberately modified in transit. However encryption algorithms don’t specialize in this work. If you (a) just want to ensure a message is not modified without bothering to encrypt it, or (b) might really want to be sure it hasn’t been messed with, then a MAC algorithm may be what you need.

Key Sizes

Some encryption algorithms only support keys with a certain “size” (eg 128-bit). Other algorithms support several different key-sizes, and some algorithms support any keysize. The actual key size used can therefore be specified in one of three different ways:

  • key size is implicit in the “cipher algorithm” string passed to Cipher.getInstance, ie the specified cipher only supports one key size
  • key size is explicit in the “cipher algorithm” string passed to Cipher.getInstance (eg “AES_128”)
  • key size is controlled by the Key object passed to Cipher.init

In the first two cases, the Key object passed to Cipher.init must match the size expected by the Cipher instance. In the last case, the Key object may be any size supported by the Cipher instance.

More on Password Based Encryption (PBE)

Password-based-encryption is the approach of having the sender and receiver of encrypted data have a shared password rather than a shared key. Keys are blocks of random numbers which are sometimes hard to transfer from one system to another - they cannot easily be ready aloud over a telephone, or memorized. However encryption algorithms are mathematical, and require “key” inputs which are large and random - something that passwords are not. The solution is to transform a password into a key via a “password based encryption key derivation function” or PBEKDF. Note that real keys are always more secure - even a good password does not have as much variety as a binary key, and a bad password of course has far less.

There are several common algorithms used for mapping a PBEKey (ie password, salt, iteration-count) into a SecretKey. The most standard ones are PBKDF#1 and PBKDF#2 (password based key derivation functions #1 and #2). However OpenSSL has its own slight variation on this which is somewhere between the two. Other encryption libraries often use adhoc (and much less secure) ways of mapping password to key.

PBE encryption can be implemented with the Java standard library as follows:

    String password = "secret";
    String salt = "salt";
    int iterationCount = 1000;

    PBEKeySpec keyspec = new PBEKeySpec(password.toCharArray(), salt.getBytes(), iterationCount);
    SecretKeyFactory skf = SecretKeyFactory.getInstance("PBEWithHmacSHA256AndAES_128");
    SecretKey key = skf.generateSecret(keyspec);
    Cipher cipher = Cipher.getInstance("PBEWithHmacSHA256AndAES_128");

    cipher.init(Cipher.ENCRYPT_MODE, key);
    String plaintext = "hello, world";
    byte[] ciphertext = cipher.doFinal(plaintext.getBytes());

    cipher.init(Cipher.DECRYPT_MODE, key);
    String recovered = new String(cipher.doFinal(ciphertext));

In this approach, no IvParameterSpec is needed by Cipher.init - the “salt” parameter associated with the key effectively solves the same problem (ensuring the same data encrypted with the same key does not generate the same output). However this only works if multiple data-blocks are encrypted with different salts - the same input encrypted with the same (password, salt) obviously produces the same output.

A minor variant to the above is to omit the salt and iteration-count params when creating the PBEKeySpec; instead instantate a PBEParameterSpec instance with these values and pass that as an additional parameter to Cipher.init. The result is the same.

Unfortunately, in this approach the SecretKey object is not yet a real key, just a wrapper around the PBEKeySpec data. The real key is derived from the SecretKey within the Cipher.init method. By design, key-derivation-functions are meant to be slow to hinder attackers (the iteration-count parameter controls exactly how slow). If your encryption use-case only requires Cipher.init to be called occasionally then this is fine. However if your use-case requires encoding many small blocks of data with the same key then that means calling Cipher.init very often which is a performance issue.

A password can be transformed explicitly into a real binary key by directly using a PBKDF algorithm as follows:

 private static final String PBKDF_ALGO = "PBKDF2WithHmacSHA256";

  private static SecretKeySpec passwordToAESKey(String password, int keylength) throws GeneralSecurityException {
    SecretKeyFactory keyFactory = SecretKeyFactory.getInstance(PBKDF_ALGO);
    PBEKeySpec keySpec = new PBEKeySpec(password.toCharArray(), SALT, NITERATIONS, keylength);
    SecretKey key = keyFactory.generateSecret(keySpec);
    byte[] tmp = key.getEncoded();
    return new SecretKeySpec(tmp, "AES");
  }

This key can then be used with a standard AES algorithm:

  private static final String AES_ALGO = "AES/CBC/PKCS5Padding";

  public String encrypt(SecureRandom secureRandom, SecretKey key, Cipher cipher, String in) {
    try {
      // Allocate a new IV for each string (equivalent to "salting" the encryption). As with salts, an IV is
      // not secret information - it is only there to make it impossible to detect when the same input has
      // been encrypted in multiple places. And the IV is needed for decryption. The IV is therefore prefixed
      // to the returned result.
      byte[] iv = new byte[16];
      secureRandom.nextBytes(iv);
      IvParameterSpec ivSpec = new IvParameterSpec(iv);

      // Initialise the instance and configure the key + ivspec + random-number-generator
      cipher.init(Cipher.ENCRYPT_MODE, key, ivSpec, secureRandom);

      byte[] data = cipher.doFinal(in.getBytes(UTF8));
      return base64Encoder.encodeToString(iv) + ":" + base64Encoder.encodeToString(data);
    } catch (GeneralSecurityException e) {
      throw new ConnectException(e);
    }
  }

Method encrypt does not repeat the KDF on each init, so it can be called efficiently:


  SecretKey key = passwordToAESKey(password, keylength); // slow, but only once
  Cipher cipher = Cipher.getInstance(AES_ALGO);

  for(String plaintext : inputs) {
    String ciphertext = encrypt(secureRandom, cipher, key, plaintext);
    ...
  }

Java Keystores

The java.security package provides quite a few classes related to ‘keystore’ functionality. Keystores are mostly used for asymmetric encryption purposes; in particular a keystore mostly holds public keys of other people and systems in order to:

  • verify the identity of a remote website when connecting to it via SSL
  • verify the identity of a signed jarfile [ie where a java security policy grants special privileges to code signed by a particular key]

When java is used to implement the server side of an SSL communication, then a keystore also holds the relevant asymmetric private key.

It is technically possible to load symmetric keys into a keystore, but that isn’t very common and there is not much benefit.

The standard Java policy and limited-strength (128-bit) encryption

Unfortunately, the standard Java distribution from Oracle (and earlier, from Sun) includes only support for 128-bit AES keys; to support longer keys a separate file must be downloaded and installed. Who is responsible for this policy isn’t clear - some sources say that the US laws limit export of strong encryption, while others say this is done because some countries ban import of software with strong encryption. Regardless, it’s a nuisance but not a major one to work around.

Quite how this policy stops us evil developers from doing the evil that we do is not clear. The restriction can be bypassed with only mild inconvenience in two ways.

Oracle provides a download link for “Java Cryptography Extension Unlimited Strength Jurisdiction Policy Files”. This is a zipfile containing two jars named “local_policy.jar” and “US_export_policy.jar”. These need to be placed into the $JAVA_HOME/jre/lib/security directory, overwriting the existing files with those names.

Alternatively, a different encryption implementation can be used, such as the one from the “bouncycastle” project, which despite the silly name and logo is very well respected. And not being based in the US (originally founded by Australians), it is not bound by these laws. Actually, although the mathematics behind good symmetric encryption algorithms is very clever, the algorithms themselves are fairly simple to implement (just a few hundred lines).

Note that when using an alternative implementation, it is not possible to register a library as a plugin “JCE Provider”, and then use the normal Java standard library APIs; the javax.crypto.Cipher class delegates to the specified “provider library” only after checking the encryption strength is allowed by the JDK policy, ie only 128-bit keys allowed unless the “unlimited strength” policy has been installed. However the bouncycastle library provides its own API that can be called directly, and this avoids all policy limitations.

Using The BouncyCastle Cryptography Library

As mentioned above, the bouncy castle project provides a library that duplicates all the encryption functionality provided in the standard JDK, and many additional algorithms. The code is open-source and of high quality. And best of all, it can be accessed via either the standard java APIs discussed above, or via its own native API.

The bouncycastle native APIs are somewhat saner than the JDK ones because they don’t funnel many algorithms through a few common SPI interfaces. And best of all, by avoiding the java.security.Cipher wrapper class, the limit of 128 bits on keylength is avoided.

Encrypt and decrypt given just a key

The following code shows how to use the BouncyCastle native API to encrypt a string given (somehow) the raw key data (not a password). A random IV is generated for use when encrypting the plaintext, and that IV is stored on the front of the resulting output (which is then base64-encoded).

Decryption of course simply base64-decodes the data, retrieves the IV from the front of the resulting byte-array, and then uses (key,iv) to decode the rest of the string.

See example.

Decrypt data that has previously been encrypted via OpenSSL using a password

The following code shows how to use the BouncyCastle native API to decrypt a string which has been encrypted using the openssl library (via its commandline, or via some program that uses the openssl libraries). Openssl uses slightly non-standard algorithms which are not supported in the JDK. It is also possible to access the bouncycastle algorithms through the standard JDK crypto API to also implement openssl compatibility, but why bother?

See example.

Decrypt data that has previously been encrypted via OpenSSL using a password

Although it is tidier to just use the BouncyCastle APIs directly, here is an example of decoding OpenSSL using BouncyCastle’s extended algorithm support but via the standard JDK APIs.

See example.

Warnings

The following code is a very common error found on internet forums:

  Key key = new SecretKeySpec(pwd.getBytes("UTF-8"), "AES");

This is using the password as the key, rather than deriving a key from the password. Keys should be fixed-size blocks of almost-random data - which passwords certainly are not. It is the role of a “key derivation function” to hash/mangle a password in such a way that it becomes a key; calling “getBytes” does not do this. See the section on “password based encryption” earlier in this document for more information.

Other Notes

The concept of random numbers (and the SecureRandom class) has been mentioned several times above; you might be interested in this article on SecureRandom.

References