1.3 Public-Key Cryptography: Key Exchange, Signing, and Post-Quantum Migration

Public-key cryptography is not simply "stronger but slower symmetric encryption." It primarily addresses three key problems: establishing a shared secret over an open network, verifying digital signatures, and enabling a sender who does not share a secret key with the receiver to securely encapsulate a key for that receiver.

Three Uses Must Be Separated

Key agreement: Each party inputs their own secret and the other party's public material → establishes a shared secret
Signature: Private key signs, public key verifies → authenticates origin and ensures integrity
KEM/Encryption: Uses recipient's public key to encapsulate a secret → recipient's private key decapsulates it

"The idea that 'using a private key to encrypt and a public key to decrypt is equivalent to signing' is a dangerous analogy. Real signature protocols perform domain separation, encoding, and hashing, and meet specific formal definitions of unforgeability. Developers should use established signing APIs (such as Ed25519, ECDSA, RSA-PSS, or standard signature protocols) rather than directly implementing low-level RSA operations."

The same key should not be used for both signing and decryption. Isolating cryptographic operations by purpose reduces attack surface, prevents cross-protocol vulnerabilities, and maintains clarity in certificate keyUsage, key permissions, and key rotation policies.

Key Negotiation Still Requires Identity Authentication

Plain Diffie–Hellman/ECDH can resist passive eavesdropping but cannot automatically prevent man-in-the-middle attacks where the attacker independently negotiates separate keys with each endpoint. TLS binds ephemeral key exchange, certificate identity, and the handshake transcript together to ensure authenticity.

Using ephemeral key agreement also provides forward secrecy: even if a long-term certificate private key is later compromised, past session traffic should not be recoverable. Forward secrecy does not guarantee permanent security, endpoints' memory, session ticket keys, and application plaintext may still be exposed.

Big Data Typically Uses Hybrid Encryption

Public-key operations are not well-suited for directly encrypting arbitrary-length files. Modern approaches first derive a shared secret via a Key Encapsulation Mechanism (KEM) or key agreement, then use a Key Derivation Function (KDF) to generate a symmetric key, which is finally used to encrypt the data with an Authenticated Encryption with Associated Data (AEAD) scheme:

recipient public key
       │ KEM encapsulation
       ▼
shared secret → KDF → AEAD key → encrypt plaintext
       │
       └── encapsulated key and ciphertext are sent together

HPKE (RFC 9180) defines a standardized combination of KEM, KDF, and AEAD with context binding. It still requires careful selection of modes, identity authentication, and message sequencing; the base mode only ensures confidentiality for the recipient and does not automatically authenticate the sender.

Signatures Verify Bytes and Context

Before signing, a normalized representation must be defined, otherwise, two systems might derive different byte sequences from the same business object. Security protocols should also bind key elements: protocol name, version, message type, tenant/environment, and validity period, to prevent signatures from being repurposed in unintended contexts.

The verification process must not rely solely on calling verify(signature, payload). It must also validate that the public key originates from a trusted relationship, that the cryptographic algorithm is permitted, that the key ID remains valid, that the message has not expired, that the nonce has not been reused, and that authorization decisions permit the subject to perform the requested action.

The security level of the signing private key depends on the failure radius. Root keys for software releases, Certificate Authority (CA) keys, and high-value transaction keys should be protected using HSMs, offline layers, dual control, or threshold schemes. Short-lived service signing keys may reside online, but should operate under minimal privileges, rotate frequently, and maintain audit trails.

Post-Quantum Cryptography Is Now in the Migration Phase

It’s uncertain when large-scale fault-tolerant quantum computers will emerge, but the principle of “encrypt now, decrypt later” means that long-term sensitive data must be acted upon well in advance. In 2024, NIST published:

• FIPS 203: ML-KEM, for establishing shared secrets;
• FIPS 204: ML-DSA, a general-purpose post-quantum digital signature;
• FIPS 205: SLH-DSA, a hash-based, stateless signature.

Key encapsulation mechanisms (KEMs) and digital signatures are not interchangeable. ML-KEM is not a general-purpose encryption API for arbitrary plaintext, and ML-DSA is not a key exchange protocol.

Migration does not mean simply replacing all existing ECC or RSA strings today. Start with a crypto inventory:

Protocol/data → Algorithm → Key location → Certificate/format → Dependency library
          → Confidentiality duration → Interoperable objects → Upgrade owner

Then design protocols and data formats to support algorithm agility, and evaluate the impact of larger public keys, ciphertexts, and signatures on MTU, handshake processes, certificates, HSMs, firmware, and logs. When deploying hybrid solutions, use standardized combinations to ensure attackers cannot downgrade to a single weak component; manually stitching together “classical keys + PQ keys” lacks automatic security guarantees.

FIPS standards may still issue errata or updated guidance, and library implementations are still rapidly maturing. Production decisions should align with protocol ecosystems and compliance requirements, relying on interoperability testing and vetted implementations, never attempting to build lattice-based cryptography from scratch.

Checklist Before Selecting a Primitive

• Is the goal key exchange, signing, KEM, or data AEAD?
• Who authenticates whom, and where do identity materials originate?
• Are forward secrecy and anti-replay protection required?
• Are keys isolated by purpose, environment, or tenant?
• Does signing bind to protocol context and use canonical encoding?
• Can algorithms and formats be upgraded without disrupting historical data?
• Is there a requirement for long-term confidentiality that necessitates starting a PQC migration now?

The next chapter connects public keys to real identities: certificate chains, service identity validation, TLS handshakes, and automatic rotation.

References

• RFC 9180, Hybrid Public Key Encryption
• NIST, FIPS 203: ML-KEM
• NIST, Post-Quantum Cryptography Publications
• RFC 9958, Post-Quantum Cryptography for Engineers