1.3 Public-Key Cryptography: Key Exchange, Signing, and Post-Quantum Migration

Public-key cryptography is not simply "stronger but slower symmetric encryption." It primarily addresses three key problems: establishing a shared secret over an open network, verifying digital signatures, and enabling a sender who does not share a secret key with the receiver to securely encapsulate a key for that receiver.

Three Uses Must Be Separated

Key agreement: Each party inputs their own secret and the other party's public material → establishes a shared secret
Signature: Private key signs, public key verifies → authenticates origin and ensures integrity
KEM/Encryption: Uses recipient's public key to encapsulate a secret → recipient's private key decapsulates it

“Signing is private-key encryption followed by public-key decryption” is a dangerous analogy. Real signature protocols perform domain separation, encoding, and hashing, and meet specific formal definitions of unforgeability. Developers should use established signing APIs (such as Ed25519, ECDSA, RSA-PSS, or standard signature protocols) rather than directly implementing low-level RSA operations.

The same key should not be used for both signing and decryption. Isolating cryptographic operations by purpose reduces attack surface, prevents cross-protocol vulnerabilities, and maintains clarity in certificate keyUsage, key permissions, and key rotation policies.

Key Agreement Still Needs Authentication

With suitable groups, key generation, and public-key validation, Diffie–Hellman/ECDH can resist passive eavesdropping but cannot automatically prevent man-in-the-middle attacks where the attacker independently negotiates separate keys with each endpoint. A certificate-authenticated TLS handshake binds key exchange, certificate identity, and the transcript; pre-shared-key modes use different authentication material.

Ephemeral key agreement, with temporary secrets properly erased after use, can provide forward secrecy: even if a long-term certificate private key is later compromised, past session traffic should not be recoverable. Forward secrecy does not guarantee permanent security, endpoints' memory, session ticket keys, and application plaintext may still be exposed.

Large Payloads Usually Use Hybrid Encryption

Public-key operations are not well-suited for directly encrypting arbitrary-length files. Modern approaches first derive a shared secret via a Key Encapsulation Mechanism (KEM) or key agreement, then use a Key Derivation Function (KDF) to generate a symmetric key, which is finally used to encrypt the data with an Authenticated Encryption with Associated Data (AEAD) scheme:

recipient public key
       │ KEM encapsulation
       ▼
shared secret → KDF → AEAD key → encrypt plaintext
       │
       └── KEM encapsulation value (not the shared secret) is sent with ciphertext

HPKE (RFC 9180) defines a standardized combination of KEM, KDF, and AEAD with context binding. It still requires careful selection of modes, identity authentication, and message sequencing; base mode provides recipient encryption and AEAD integrity but does not authenticate the sender. Anyone with the recipient’s public key can create a new ciphertext. Later compromise of the static recipient private key may also expose recorded base-mode encapsulations; using HPKE alone does not imply forward secrecy.

Signatures Verify Bytes and Context

A signing protocol must define exact bytes, either through a canonical encoding or by transmitting and verifying the original bytes. Reserializing the “same business object” can change field order, whitespace, or number formatting and cause verification to fail. Security protocols should also bind key elements: protocol name, version, message type, tenant/environment, and validity period, to prevent signatures from being repurposed in unintended contexts.

The verification process must not rely solely on calling verify(signature, payload). It must also validate that the public key originates from a trusted relationship, that the cryptographic algorithm is permitted, that the key ID remains valid, that the message has not expired, that the nonce has not been reused, and that authorization decisions permit the subject to perform the requested action.

The security level of the signing private key depends on the impact of compromise. Root keys for software releases, Certificate Authority (CA) keys, and high-value transaction keys should be protected using HSMs, offline layers, dual control, or threshold schemes. Short-lived service signing keys may reside online, but should operate under minimal privileges, rotate frequently, and maintain audit trails.

Post-Quantum Cryptography Is Now in the Migration Phase

It’s uncertain when large-scale fault-tolerant quantum computers will emerge, but the principle of “harvest now, decrypt later” means that long-term sensitive data must be acted upon well in advance. In 2024, NIST published:

• FIPS 203: ML-KEM, for establishing shared secrets;
• FIPS 204: ML-DSA, a general-purpose post-quantum digital signature;
• FIPS 205: SLH-DSA, a hash-based, stateless signature.

Key encapsulation mechanisms (KEMs) and digital signatures are not interchangeable. ML-KEM is not a general-purpose encryption API for arbitrary plaintext, and ML-DSA is not a key exchange protocol.

Migration does not mean simply replacing all existing ECC or RSA strings today. Start with a crypto inventory:

Protocol/data → Algorithm → Key location → Certificate/format → Dependency library
          → Confidentiality duration → Interoperability peers → Upgrade owner

Then design protocols and data formats to support algorithm agility, and evaluate the impact of larger public keys, ciphertexts, and signatures on MTU, handshake processes, certificates, HSMs, firmware, and logs. When deploying hybrid solutions, use standardized combinations to ensure attackers cannot downgrade to a single weak component; manually stitching together “classical keys + PQ keys” lacks automatic security guarantees.

These standards are final, but library support, applicable implementation validation, and protocol-combination status require separate checks. FIPS publications may receive errata or updated guidance, and library implementations are still rapidly maturing. Production decisions should align with protocol ecosystems and compliance requirements, relying on interoperability testing and vetted implementations, never attempting to build lattice-based cryptography from scratch.

Checklist Before Selecting a Primitive

• Is the goal key exchange, signing, KEM, or data AEAD?
• Who authenticates whom, and where do identity materials originate?
• Are forward secrecy and anti-replay protection required?
• Are keys isolated by purpose, environment, or tenant?
• Does signing bind to protocol context and use canonical encoding?
• Can algorithms and formats be upgraded without disrupting historical data?
• Is there a requirement for long-term confidentiality that necessitates starting a PQC migration now?

Continue with X.509 and PKI to bind public keys to service identities, then study TLS handshakes and automatic rotation.

References

• RFC 9180, Hybrid Public Key Encryption
• NIST, FIPS 203: ML-KEM
• NIST, FIPS 204: ML-DSA
• NIST, FIPS 205: SLH-DSA
• RFC 9958, Post-Quantum Cryptography for Engineers