Merge branch 'openpgp-smartcard-decryption' into 'main'

Implement support for OpenPGP External Secret Keys

See merge request root/bc-java!305

Author: Administrator

CONTRIBUTORS.html

@@ -450,7 +450,7 @@
 <li>Adam Vartanian &lt;https://github.com/flooey&gt; use of ShortBuffer exception and buffer size pre-check in Cipher.doFinal().</li>
 <li>Bernd &lt;https://github.com/ecki&gt; Fix to make PGPUtil.pipeFileContents use buffer and not leak file handle.</li>
 <li>Shartung &lt;https://github.com/shartung&gt; Additional EC Key Agreement algorithms in support of German BSI TR-03111.</li>
-<li>Paul Schaub &lt;https://github.com/vanitasvitae&gt; bringing PGPSecretKey.getUserIds() into line with PGPPublicKey.getUserIds(). Exception message fix in BcPublicKeyDataDecryptorFactory. Additional tests on PGP key ring generation. Improved functionality of PGPSignatureSubpacketGenerator, PGPPublicKeyRing. Tweaks to PGPDataEncryptorBuilder interface, fix for JcaPGP/BcPGP Ed25519 private key conversion. Added configurable CRC detection to ArmoredInputStream, additional control character skipping in ArmoredInputStream. Rewind code for PGPPBEEncryptedData, addition of PGPSignature.getDigestPrefix(). Wrong list traversal fix in PGPSecretKeyRing. Further improvement to use of generics in PGP API. General interop improvements. PGP Public / Secure keyring ignore marker packets when reading. Initial work on PGP session key handling, filtering literal data for canoncialization. Addition of direct key identified key-ring construction. PGPSecretKeyRing.insertOrReplacePublicKey addition. Addition of utility methods for joining/merging signatures and public keys. Addition of PGP regexp packet, PolicyURI packet handling, UTF8 comment testing. Efficiency improvements to TruncatedStream. Initial Argon2 support for OpenPGP. General cleanups. Fast CRC24 implementation, SHA3 addtions to BcImplProvider, improvements to One Pass Signature support, signatue validation, read() consistency in BCPGInputStream. Contributions to AEAD support (v6 & v5) in PGP API. Addition of PGP WildCard ID, moving the PGP example code into the 21st century. Security patches for encrypted data generation, initial thread safe certification verification. Support for V6 EC keys, V6 signatures, V6 encryption, V6 PKESK, PGP packet criticality, and Preferred AEAD CipherSuites sigsubpacket support. Introduce high-level OpenPGP API for message creation/consumption and certificate evaluation. OpenPGP fuzz testing. Fix to prevent a null pointer exception on processing a partial stripped key. Moving the Argon2 memory size exponent bounds check from S2K packet parsing to decryption time. ArmoredInputStream CSF dash-escape hardening. Report and initial patch for OnePassSignaturePacket defaulting to the Legacy packet format for v6 packets (github #2347). PGPKeyPairGenerator factory methods for the OpenPGP brainpool curves (github #2375).</li>
+<li>Paul Schaub &lt;https://github.com/vanitasvitae&gt; bringing PGPSecretKey.getUserIds() into line with PGPPublicKey.getUserIds(). Exception message fix in BcPublicKeyDataDecryptorFactory. Additional tests on PGP key ring generation. Improved functionality of PGPSignatureSubpacketGenerator, PGPPublicKeyRing. Tweaks to PGPDataEncryptorBuilder interface, fix for JcaPGP/BcPGP Ed25519 private key conversion. Added configurable CRC detection to ArmoredInputStream, additional control character skipping in ArmoredInputStream. Rewind code for PGPPBEEncryptedData, addition of PGPSignature.getDigestPrefix(). Wrong list traversal fix in PGPSecretKeyRing. Further improvement to use of generics in PGP API. General interop improvements. PGP Public / Secure keyring ignore marker packets when reading. Initial work on PGP session key handling, filtering literal data for canoncialization. Addition of direct key identified key-ring construction. PGPSecretKeyRing.insertOrReplacePublicKey addition. Addition of utility methods for joining/merging signatures and public keys. Addition of PGP regexp packet, PolicyURI packet handling, UTF8 comment testing. Efficiency improvements to TruncatedStream. Initial Argon2 support for OpenPGP. General cleanups. Fast CRC24 implementation, SHA3 addtions to BcImplProvider, improvements to One Pass Signature support, signatue validation, read() consistency in BCPGInputStream. Contributions to AEAD support (v6 & v5) in PGP API. Addition of PGP WildCard ID, moving the PGP example code into the 21st century. Security patches for encrypted data generation, initial thread safe certification verification. Support for V6 EC keys, V6 signatures, V6 encryption, V6 PKESK, PGP packet criticality, and Preferred AEAD CipherSuites sigsubpacket support. Introduce high-level OpenPGP API for message creation/consumption and certificate evaluation. OpenPGP fuzz testing. Fix to prevent a null pointer exception on processing a partial stripped key. Moving the Argon2 memory size exponent bounds check from S2K packet parsing to decryption time. ArmoredInputStream CSF dash-escape hardening. Report and initial patch for OnePassSignaturePacket defaulting to the Legacy packet format for v6 packets (github #2347). PGPKeyPairGenerator factory methods for the OpenPGP brainpool curves (github #2375). Support for OpenPGP External Secret Keys, and the initial OpenPGP smart card API (bcpgsc) with YubiKey and simulator backends (github #2339, #2374).</li>
 <li>Nick of Nexxar &lt;https://github.com/nros&gt; update to OpenPGP package to handle a broader range of EC curves.</li>
 <li>catbref &lt;https://github.com/catbref&gt; sample implementation of RFC 7748/Ed25519 (incorporated work from github users Valodim and str4d as well).</li>
 <li>gerlion &lt;https://github.com/gerlion&gt; detection of concurrency issue with pre-1.60 EC math library.</li>

build.gradle

@@ -316,7 +316,7 @@ configure(subprojects.findAll {it.name != 'bom'}) {
 }
 
 
-test.dependsOn([':core:test', ':prov:test', ':prov:test11', ':prov:test15', ':prov:test17', ':pkix:test', 'pg:test', ':tls:test', ':tls-klog:test', 'mls:test', 'mail:test', 'jmail:test'])
+test.dependsOn([':core:test', ':prov:test', ':prov:test11', ':prov:test15', ':prov:test17', ':pkix:test', 'pg:test', ':pgsc:test', ':tls:test', ':tls-klog:test', 'mls:test', 'mail:test', 'jmail:test'])
 
 // Aggregate all published jars (main, sources, javadoc) into a top-level dist/
 // directory so consumers have a single place to pick up the build outputs.

docs/releasenotes.html

@@ -83,6 +83,8 @@ <h3>2.1.2 Defects Fixed</h3>
 
 <h3>2.1.3 Additional Features and Functionality</h3>
 <ul>
+<li>OpenPGP secret keys whose private key material is held outside the key - on a hardware token - are now supported, following <a href="https://datatracker.ietf.org/doc/draft-dkg-openpgp-external-secrets/">draft-dkg-openpgp-external-secrets</a>. org.bouncycastle.bcpg.SecretKeyPacket recognises the External S2K usage octet (SecretKeyPacket.USAGE_EXTERNAL) and its locator hint, PGPSecretKey exposes isExternalKey() / getExternalKeyLocatorHint(), and a decryption backend is plugged into the high level API by registering an org.bouncycastle.openpgp.api.PublicKeyDataDecryptorFactoryProvider with OpenPGPMessageProcessor. On the lightweight side BcPublicKeyDataDecryptorFactory now isolates the raw private-key operation behind BcPublicKeyCryptoCallback, so a subclass of the new BcExternalPublicKeyDataDecryptorFactory routes only the RSA decryption or the ECDH/X25519 agreement to a device while inheriting all packet parsing, KDF and key-unwrap logic. Note the External S2K usage octet is provisional: the draft records it as "TBD (252?)" and IANA has not assigned it.</li>
+<li>A new module, bcpgsc (org.bouncycastle.openpgp.smartcard), provides an OpenPGP smart-card API on top of the external-secret-key support: listing cards across pluggable backends, uploading key material, and decrypting messages with a card-held key. Two backends ship - a YubiKey backend built on the YubiKit libraries, and an in-memory simulator for testing without hardware. The YubiKit libraries are a compile-only dependency, so the published bcpgsc jar carries no third-party runtime dependency; an application using org.bouncycastle.openpgp.smartcard.yubikey must add them to its own classpath. Contributed by Paul Schaub.</li>
 <li>org.bouncycastle.openpgp.operator.PGPKeyPairGenerator gained named convenience methods for the three brainpool curves RFC 9580 sec. 9.2 permits for OpenPGP - generateBrainpoolP256r1ECDHKeyPair / generateBrainpoolP384r1ECDHKeyPair / generateBrainpoolP512r1ECDHKeyPair and the matching ECDSA variants - alongside the existing NIST P-256/P-384/P-521 methods. The generated ECDH keys carry the per-curve KDF hash and KEK symmetric algorithm required by sec. 11.5.1, which the key-pair generator tests now assert for the brainpool and the NIST curves alike. Contributed by Paul Schaub.</li>
 <li>Further extended key usages are now available as KeyPurposeId constants: id-kp-secureShellClient and id-kp-secureShellServer (RFC 6187 sec. 2.2.2, id-kp 21/22), id-kp-cmcArchive (RFC 6402 sec. 2.10, id-kp 29) and id-kp-bundleSecurity (RFC 9174, id-kp 35).</li>
 <li>The two RFC 4556 (PKINIT) extended key usages are now available as KeyPurposeId constants: id-pkinit-KPClientAuth (1.3.6.1.5.2.3.4, sec. 3.2.2) as KeyPurposeId.id_kp_pkinitClientAuth and id-pkinit-KPKdc (1.3.6.1.5.2.3.5, sec. 3.2.4) as KeyPurposeId.id_kp_pkinitKdc. Note these sit under the Kerberos id-pkinit arc rather than the PKIX id-kp arc, and are distinct from the Microsoft smartcard logon usage already available as id_kp_smartcardlogon.</li>

pg/src/main/java/org/bouncycastle/bcpg/PublicKeyUtils.java

@@ -1,5 +1,7 @@
 package org.bouncycastle.bcpg;
 
+import org.bouncycastle.asn1.cryptlib.CryptlibObjectIdentifiers;
+
 /**
  * Utility methods related to OpenPGP public key algorithms.
  */
@@ -52,4 +54,33 @@ public static boolean isEncryptionAlgorithm(int publicKeyAlgorithm)
             return false;
         }
     }
+
+    /**
+     * Return true, if the passed in {@link PublicKeyPacket} is based on X25519, either the legacy variant
+     * via {@link PublicKeyAlgorithmTags#ECDH} over curve25519, or the modern
+     * {@link PublicKeyAlgorithmTags#X25519}.
+     *
+     * @param publicKeyPacket public key packet
+     * @return true if the key is an X25519 key
+     */
+    public static boolean isX25519Key(PublicKeyPacket publicKeyPacket)
+    {
+        int algorithm = publicKeyPacket.getAlgorithm();
+        if (algorithm == PublicKeyAlgorithmTags.X25519)
+        {
+            return true;
+        }
+        if (algorithm != PublicKeyAlgorithmTags.ECDH)
+        {
+            return false;
+        }
+        // the algorithm tag and the key packet body can disagree on malformed input, so type-check
+        // rather than cast: an ECDH tag over a non-EC body is simply not an X25519 key.
+        BCPGKey key = publicKeyPacket.getKey();
+        if (!(key instanceof ECPublicBCPGKey))
+        {
+            return false;
+        }
+        return CryptlibObjectIdentifiers.curvey25519.equals(((ECPublicBCPGKey)key).getCurveOID());
+    }
 }

pg/src/main/java/org/bouncycastle/bcpg/SecretKeyPacket.java

@@ -5,6 +5,7 @@
 import java.io.IOException;
 
 import org.bouncycastle.util.Arrays;
+import org.bouncycastle.util.io.Streams;
 
 /**
  * Base class for OpenPGP secret (primary) keys.
@@ -63,14 +64,42 @@ public class SecretKeyPacket
      * Users should migrate to AEAD with all due speed.
      */
     public static final int USAGE_AEAD = 0xfd;
-    
+
+    /**
+     * Externally-backed secret key material.
+     * S2K-usage octet indicating that the secret key material is stored externally, e.g. on a hardware device.
+     * The draft specification is an alternative to GnuPGs proprietary {@link S2K#GNU_DUMMY_S2K} mechanism.
+     * <p>
+     * NOTE: this code point is <em>provisional</em>. draft-dkg-openpgp-external-secrets-03 sec. 2 records
+     * it as "TBD (252?)" and IANA has not yet assigned it, so the value here tracks the draft's suggestion
+     * and is subject to change if a different octet is allocated. Do not rely on it for long-term storage.
+     *
+     * @see <a href="https://datatracker.ietf.org/doc/draft-dkg-openpgp-external-secrets/">
+     *     OpenPGP External Secret Keys</a>
+     */
+    public static final int USAGE_EXTERNAL = 0xfc;
+
+    /**
+     * Maximum accepted length of the external key locator hint of a version 4 secret key packet, whose
+     * hint is not length-prefixed and so is bounded only by the packet. Mirrors
+     * {@link SignaturePacket#MAX_SUBPACKET_LEN}.
+     */
+    public static final int MAX_EXTERNAL_LOCATOR_HINT_LEN = 2 * 1024 * 1024;
+
+    /**
+     * Maximum length of the external key locator hint of a version 5 or 6 secret key packet, whose
+     * conditional parameters are prefixed with a one-octet count (RFC 9580 sec. 5.5.3).
+     */
+    public static final int MAX_V6_EXTERNAL_LOCATOR_HINT_LEN = 255;
+
     private PublicKeyPacket pubKeyPacket;
     private byte[] secKeyData;
     private int s2kUsage;
     private int encAlgorithm;
     private int aeadAlgorithm;
     private S2K s2k;
     private byte[] iv;
+    private byte[] externalKeyLocatorHint;
 
     /**
      * Parse a primary OpenPGP secret key packet from the given OpenPGP {@link BCPGInputStream}.
@@ -140,13 +169,33 @@ public class SecretKeyPacket
         s2kUsage = in.read();
 
         int conditionalParameterLength = -1;
-        if (version == PublicKeyPacket.LIBREPGP_5 || 
+        if (version == PublicKeyPacket.LIBREPGP_5 ||
            (version == PublicKeyPacket.VERSION_6 && s2kUsage != USAGE_NONE))
         {
             // TODO: Use length to parse unknown parameters
             conditionalParameterLength = in.read();
         }
 
+        if (s2kUsage == USAGE_EXTERNAL)
+        {
+            if (conditionalParameterLength >= 0)
+            {
+                // v5/v6 carry an explicit count of the conditional parameters, which for an external
+                // key is exactly the locator hint - honour it, rather than reading to end of stream.
+                externalKeyLocatorHint = new byte[conditionalParameterLength];
+                in.readFully(externalKeyLocatorHint);
+            }
+            else
+            {
+                // v4 has no count: the hint is the remainder of the packet (draft sec. 2). Bound the
+                // read - a partial-length packet body shares the underlying stream (BCPGInputStream
+                // hands back "this"), so an unbounded readAll() would consume the rest of the input
+                // and turn a short header into an arbitrary allocation.
+                externalKeyLocatorHint = Streams.readAllLimited(in, MAX_EXTERNAL_LOCATOR_HINT_LEN);
+            }
+            return;
+        }
+
         if (s2kUsage == USAGE_CHECKSUM || s2kUsage == USAGE_SHA1 || s2kUsage == USAGE_AEAD)
         {
             encAlgorithm = in.read();
@@ -205,7 +254,7 @@ public class SecretKeyPacket
                     if (encAlgorithm < 7)
                     {
                         iv = new byte[8];
-                    } 
+                    }
                     else
                     {
                         iv = new byte[16];
@@ -214,7 +263,7 @@ public class SecretKeyPacket
                 }
             }
         }
-        
+
         if (version == PublicKeyPacket.LIBREPGP_5)
         {
             long keyOctetCount = ((long) in.read() << 24) | ((long) in.read() << 16) | ((long) in.read() << 8) | in.read();
@@ -233,6 +282,59 @@ public class SecretKeyPacket
         }
     }
 
+    /**
+     * Create a SecretKeyPacket representing an external secret key ({@link #USAGE_EXTERNAL}).
+     *
+     * @see <a href="https://datatracker.ietf.org/doc/draft-dkg-openpgp-external-secrets/">
+     *     OpenPGP External Secret Keys</a>
+     * @param pubKeyPacket public key packet
+     * @param locatorHint optional external key locator hint
+     */
+    public SecretKeyPacket(
+            PublicKeyPacket pubKeyPacket,
+            byte[] locatorHint)
+    {
+        this(SECRET_KEY, pubKeyPacket, locatorHint);
+    }
+
+
+    /**
+     * Create a SecretKeyPacket representing an external secret key ({@link #USAGE_EXTERNAL}).
+     *
+     * @see <a href="https://datatracker.ietf.org/doc/draft-dkg-openpgp-external-secrets/">
+     *     OpenPGP External Secret Keys</a>
+     * @param keyTag key packet type
+     * @param pubKeyPacket public key packet
+     * @param locatorHint optional external key locator hint
+     */
+    protected SecretKeyPacket(
+            int keyTag,
+            PublicKeyPacket pubKeyPacket,
+            byte[] locatorHint)
+    {
+        this(keyTag, pubKeyPacket, 0, 0, USAGE_EXTERNAL, null, null, null);
+
+        byte[] hint = locatorHint == null ? new byte[0] : Arrays.clone(locatorHint);
+        int version = pubKeyPacket.getVersion();
+        if (version == PublicKeyPacket.LIBREPGP_5 || version == PublicKeyPacket.VERSION_6)
+        {
+            // a v5/v6 secret key prefixes its conditional parameters - here the locator hint - with a
+            // one-octet count, so a longer hint could not be encoded: the count would wrap while the
+            // hint was still written in full, producing a packet that does not round-trip
+            if (hint.length > MAX_V6_EXTERNAL_LOCATOR_HINT_LEN)
+            {
+                throw new IllegalArgumentException("external key locator hint too long for a version "
+                    + version + " secret key: " + hint.length + " > " + MAX_V6_EXTERNAL_LOCATOR_HINT_LEN);
+            }
+        }
+        else if (hint.length > MAX_EXTERNAL_LOCATOR_HINT_LEN)
+        {
+            throw new IllegalArgumentException("external key locator hint too long: " + hint.length
+                + " > " + MAX_EXTERNAL_LOCATOR_HINT_LEN);
+        }
+        this.externalKeyLocatorHint = hint;
+    }
+
     /**
      * Construct a {@link SecretKeyPacket}.
      * Note: <pre>secKeyData</pre> needs to be prepared by applying encryption/checksum beforehand.
@@ -426,6 +528,24 @@ public byte[] getSecretKeyData()
         return secKeyData;
     }
 
+    /**
+     * If the key has external private key material (s2k usage {@link #USAGE_EXTERNAL}), return the locator hint data.
+     * If the locator hint is empty, it is referred to as "best effort".
+     * Otherwise, the first octet indicates the type of locator hint.
+     *
+     * @see <a href="https://www.ietf.org/archive/id/draft-dkg-openpgp-external-secrets-03.html#name-openpgp-external-secret-key">
+     *     OpenPGP External Secret Key Locator Hint type registry</a>
+     * @return locator hints data
+     */
+    public byte[] getExternalKeyLocatorHint()
+    {
+        if (s2kUsage != USAGE_EXTERNAL)
+        {
+            return null;
+        }
+        return Arrays.clone(externalKeyLocatorHint);
+    }
+
     /**
      * Return the encoded packet content without packet frame.
      * @return encoded packet contents
@@ -443,7 +563,7 @@ public byte[] getEncodedContents()
 
         // conditional parameters
         byte[] conditionalParameters = encodeConditionalParameters();
-        if (pubKeyPacket.getVersion() == PublicKeyPacket.LIBREPGP_5 || 
+        if (pubKeyPacket.getVersion() == PublicKeyPacket.LIBREPGP_5 ||
            (pubKeyPacket.getVersion() == PublicKeyPacket.VERSION_6 && s2kUsage != USAGE_NONE))
         {
             pOut.write(conditionalParameters.length);
@@ -474,6 +594,12 @@ public byte[] getEncodedContents()
     private byte[] encodeConditionalParameters()
         throws IOException
     {
+        if (s2kUsage == USAGE_EXTERNAL)
+        {
+            // for an external key the conditional parameters are exactly the locator hint
+            return Arrays.clone(externalKeyLocatorHint);
+        }
+
         ByteArrayOutputStream conditionalParameters = new ByteArrayOutputStream();
         boolean hasS2KSpecifier = s2kUsage == USAGE_CHECKSUM || s2kUsage == USAGE_SHA1 || s2kUsage == USAGE_AEAD;