(With severe apologies to Miles Davis.)
Post-Quantum Cryptography is coming. But in their haste to make headway on algorithm adoption, standards organizations (NIST, IETF) are making a dumb mistake that will almost certainly bite implementations in the future.
Sophie Schmieg wrote about this topic at length and Filippo Valsorda suggested we should all agree to only use Seeds for post-quantum KEMs. You can read their words and come to the same conclusion as what I’ve written here, but I thought this was important enough to write about–if, for no other reason, to make some cryptographer shitposts make more sense to everyone else. (Nobody likes to feel excluded from inside jokes, after all!)
OwO, What’s This?
It’s not at all necessary to understand advanced topics in cryptography to appreciate the current dilemma. In fact, the underlying algorithm really doesn’t matter at the moment. So let’s set aside the technical topics for a moment. We can get back to them in a later section.
Instead, imagine you have some algorithm that does nightmare magic math that cares what color pen you use. We’ll call it Whistle for now, since my fursona is also called a whistling dog sometimes.
Because of the nightmare magic math, you might decide to implement Whistle to accept a large, expanded secret key. At first glance, this isn’t a crazy decision: A pure mathematics description of our algorithm calls for such a monstrosity. These keys are kilobytes in size.
But that can be both inconvenient and unwieldy. So you might want to allow shorter keys to be stored, and then use those to derive the larger, expanded keys.
In cryptography terms, we call the shorter keys used to derive the other secret parameters in a complex algorithm a “seed”.
Ultimately, regardless of what Whistle does, or even in, the fact that you have two ways to implement it, with different kinds of inputs, means they are actually two different algorithms.
func WhistleWithSeed(seed [32]byte, ciphertext []byte) ([]byte, error) {
expandedKey := ExpandKey(seed)
return WhistleInternal(expandedKey, ciphertext)
}
func WhistleWithExpanded(expandedKey, ciphertext []byte) ([]byte, error) {
// Hope this is done in constant-time:
valid, err := ValidateKey(expandedKey)
if err != nil {
return nil, err
}
if !valid {
return nil, errors.New("invalid key")
}
return WhistleInternal(expandedKey, ciphertext)
}
Unfortunately, standards organizations decided that these two algorithms are actually the same algorithm.
Whether this is because they never read The Art of Computer Programming by Donald Knuth (and therefore never learned what the definition for an algorithm is), or they’re stretching the definition for the sake of convenience, is difficult to say.
And, sure, there are some convenient reasons to ignore this definition:
- The same underlying algorithm (
WhistleInternalin the pseudocode above) gets called either way. - Whether you chose to store seeds or expanded keys, you’re going to get the same public key out of the other end of key generation.
- Similarly, the actual operation of
WhistleInternal(the underlying subroutine) will produce the same set of {correct, incorrect} results for the same inputs.
But the downside is that software that expects one input (a seed or an expanded key), when given the other type of input, may misbehave in a way that attackers find useful.
The Keys You Don’t Play
Imagine you’re transfering a private key from one system (that uses a cryptography library which expects expanded keys) to another (that expects seeds).
Note: You generally don’t do such a thing. Private keys should never leave the device they’re generated on for most applications. However, there are corner cases where you need to do this.
The encoding rules for storing private keys to disk are generally defined in PKCS8 (and use heavily in ASN.1). Each algorithm is supposed to have its own OID (object identifier), which is never reused for different algorithms.
In order to shoehorn two different algorithms as the same algorithm, you generally have to encode these keys using something like ASN.1’s CHOICE. Unfortunately, NULL is always a valid CHOICE.
As Sophie noted in How not to format a private key:
It is easy to imagine one part of a system writing one field, and the other part of the system reading the other. In particular, since the seed is unrecoverable after the fact, a system that only has the semi-expanded key available might, instead of writing NULL in the seed field, write “NULL”, which the other system then prefers over the semi-expanded key that contains the actual key material, both compromising the system (as “NULL” is very brute-forcable) and potentially losing the actual private key forever.
And that’s the danger inherent in these standards organizations’ insistence on supporting both seeds and semi-expanded private keys.
What Should We Do?
As Filippo suggests, we should all agree to only use seeds.
He specifically said that about ML-KEM, but I think we should apply it to all post-quantum algorithms (i.e., ML-DSA as well, at the very least).
The semi-expanded key format should only be tolerable as a runtime artifact of expanding the key from a seed, and the seed is the only thing that anyone should ever write to disk.
- For ML-KEM, seeds are 64 bytes (512 bits) generated from a secure random source.
- Half of the seed is actually used for decapsualtion, the other half is for rejecting invalid inputs without side-channels.
- For ML-DSA, seeds are 32 bytes (256 bits) of randomness.
- Do not confuse seeds for “seed phrases” (a term used with cryptocurrency wallets).
- For example, compare the
seedwithskey(secret key) entries in the “Known Anaswer Tests” for ML-DSA.
seed:
7c9935a0b07694aa0c6d10e4db6b1add2fd81a25ccb148032dcd739936737f2d
secret key:
dc7bc9a2e0b6dc66823ae4fbde971c0cfc46f9d96bbfbeebb3470ae0a5a0139f
f037b84e75537e0a1cf02a517acfe323ffffe11df72e4f38430e0e66a2654b2f
2ef757da47649d9f63fa03f1bf6fe6bc7c62971a98a2bd9d36eb0ec43ad4e9d9
40df3bb5874f5c92192aa31e0535d3cf70950bba858d11a688eaf854f63ecfc5
20c50d624891434265d8b0680c03061040299a104082c0910c8508d1100d44a6
509408292211125b90508a2688e1302dc4021280028ac302611820851237808a
000ae2040421b4910bb80550a08051b2511c28428a3672a494504910201bb451
61424424a75001328181942d62a850023449ca94200b296213156408924c4812
2100b605030208e0060200a311e1802021116483a62898029291480801083041
066613200e5b360951400c53000aa08851944842e316704ab2089b9244002512
1b0309418209c2a0800b290a819851c4340da4424500a0105b048e6034001389
28a4422648002c90202d194068e2146d19278a083746e4146914006422c660d3
a03013242844965014166da0284dcc462e94367100232e1c114909a204013106
0a2172c2142ada000c5a260d13228a62c444e3142d013445980224d33841c030
8121a621e348720b1984d2c89108b8690887714a2884d496451a9301ca2285da
30859ac851dcc00820106060465262302aa224251044640b2842988011540692
144251d236719bb4900b082890188e41c469e1a469032160e01409d3020c20c8
8c1cb23164086218476920228ccb847008952802955053327001340588842454
1041d202881aa84ccac88181008d0392899ab809d9900c9a1290614065c9322d
89860c123521cc4266c8360010062411028ea3b44d44023043a0285a002ed198
0c4882658922441c010212907084226e12134d011902519064113364c91806c2
c04589262908b63024308cda022e0c27250b367058162c5116420b4946c12088
41246c99466a04434e18a86c821661922028639409c30211029520211782d438
68003460c84688e0160000a32dc0a82824b640831464c81022a2086503234ac8
122ea098418c2072cc308a62c665093408412682da4290893285149670812260
01176d5948428ab88d592051d80892e2c0889044700ac0245a020904218a59c4
5094441094140820460209270c441020dcc8209212015038250c456e4a166622
3770dc808ca426412222441ba3618a343099844099c42952046d88146ccb242a
7cd129a8d333115c62d033b6a8357cf7cd10268ab12f16fceb7975d0a28a6c48
22213c9a772df084ad91a669e2040550fc5e8d0aeb10fab2375fc9625ef9cd48
c19631997a1cb6455d2c6286c569c9637add0317ce990996b28e51c3f3f717fb
5907bbdd53961ad3497f2c3c473cce170906ac4c624a89aa8fbe624d99385e9c
9548bf05e8cafd47d2476e41b73001f813726499e88b2b3b6f596ca311657850
346598994c40e34747161e4e76264deef2a3019389d1594c942301af47b7544c
23ecda2df2dece81e487d8f3f58ea89cd811d7275807ff1b0369ba86470088c1
74a3099fdafbe5fbb4d158801053b2b435d54059e26dee76d10a7a372f06b0b8
8b985b32f52052387438be8dc8bc6ae7369e2da9aa5e2585f8de403d091ccb7f
790d54ddb34c608b0876f2825e9113be20a2b85867a01bda53287ac780bcd8b6
06d2e6d7712c56ce0142d22fe6b786de544963e134fecedfafb83d763061d799
096a59e30d4472e440ae1faaabdf42640ce69740ceb9cae1a9612c21931b74af
3f780236123321b205b6efd6cbb134f4c73d63c0c13e660b59d5920bc33197c3
55853d8d1cddc7959f7bc500ac81d985016f5b89a0eec79b0d9364ead8e38577
c2a6549f2d067cb09438fdb21220aec80f6e22a476f332a2a4a0b7acbeb9e078
d2b5a92ae84c924f7cb19fc7df377beb6546af97aa985c747cd111a127a674b4
c26d89c14485b82e3a498a12d05406febd6c4d4b8bc051ab2cb91224b0785383
74b794b7dd9ddf3ac2b4a671fb7b9cf5acb78622ae2709eb2db16943aa24a9c9
7a81077bc784d25c0ea5991d2de883798a1f0e78f3361ed6a10dded81b1d6836
58331534fd7c01bc0eb00dfc4c3c84f0693046ff806bb200dd7bd4c0e6abca3f
2934b4814fc0e1f8be615a2dda7c8a8d06cf9ce8566b40f4a6543b25bacddc92
6863fc0fa2007d6d7bf6d18dc98df696bd0865bf0be4c492b8043a32def8e359
5ba7da345252f38f95be10fd7fb899b498fa01b09de5d5608eabc44a721aa04c
4ef1dcb86102ac5f5f79c9708dcf5c5e896edd8c2c7bde3fa83e6ffce22d6617
4e31657a0b6361585e669d3031952f08631ae1f16ff90b90d0aad3c6d7e1dd0a
9c41ab00a6e1c4f96af9ac5b79fcf821ffc016cb059245fb78dbe6c633d965aa
ab5333be07195c4b74b18e4600ce783c0a914ef4281016e80a7c9aa92d0fd789
879c5e6751125ecb154432311e41cebd4fab3a31e4d2ce22d0f8c67737bf8a0d
d85fe1349d5079a4d5feb3fee9378ca47ae46cc58a3f02038cfd53c4cee9cc42
70cebc3d115a39c831e8ed41c4dbe4051b51d7872ba0c2bb163e0085201188ea
a624a6bea9400a3a1fcc355a57f15704e61fda55a5dbaea8448fa5cb2d377a07
f58305ad107e844ab4806e5bf99c1f513ee1d0a2acc04549f0801742169a7797
1d0adbfbfe0dd2ee5d16bc461e35748d1f3f6f4598321e8c49e79e740f990359
858d2729dde007fcb26fdda9aa6e2ec4bd736f2836e7e4c83440191c849f6a53
c72a4f8f830d001ea3b18f3cb4a5bd3cf066032b4932cfd2e62a9b55723fa61c
688c935518af6860cd649bfbf1bf5fdc1f36dcaefaa157438d1cc8d56a150161
511df82631f5e88e773e4ce263f276b7b3678d4c6fc75311d411c0d01bfdb595
bb70552838e1b86517c837d909e772b428599e1fe569f77ce61531fde6fd31cd
ce1bdee4ba467fcbfbb9feeaad99fef67d4906e036c73662ddce158d4e5d4635
e5d366f79f31a19d1b3dc4a591b0df194bb06c18147f41d88d1a409becdfb67e
b063d16312266fd51b521ba9115e2e5e2aeae6ec511cede13ed4132ffbe0273f
6c7039b3874f058804a54809af60557a21d9b4b831d04156a7c22dcbcdfe14f6
2437f449cb5ef12bf4251d485496cd835c0c2bc58bd845963dfa76ecd68519c4
bdaf110be7ab052876dc3407591568c956ea3bf107c90fd5853a292f59a8d4b5
8b5d3fddf29bdbeac36852e3c69766fe460176a801831292b8e88a74a01ecbbe
09a7b4d74cfd7fd628841944d9d556dbd60c76f96f07dc53443805ee9aa09365
de4fb8179252c6b099b5dd351fdefc23dbd8090596c5d208ffd2c5661d8e5612
dd574fc69045c769a969e600d77cfe192f1d3ae911289355c585811491b0ccd7
3692ab158824ab9edf8ac8193f0b33e6138b72c6dcd5d344f807b3da92425037
de5ea4eead1c795effaa145e2ecdd327606eb2609929b9474b2bb04653602555
c068385e92f06f29ca613ce5b4404f01ab1805db0acaa890330d291f40692df3
82509302b6dc8668f2c8f2d3a44fd58dca26e9802794f73d25b3149e6d576441
Standards organizations will continue to insist on supporting use cases where people store and pass around semi-expanded private keys, rather than seeds, and cite the performance hit of deriving the semi-expanded key from a seed as a reason to tolerate such behavior.
But standards organizations cannot force the rest of us to support these features in our actual software implementations of the standards. So let’s opt to never do that, where reasonably practical.
Sophie suggested that seeds and semi-expanded keys should be two different OIDS (in PKCS8 parlance). That’s a good compromise for systems that, for whatever stupid reason, can’t commit to only permitting seeds as the private key format for post-quantum cryptography.
If you’re writing software the expects a seed and you get a semi-expanded private key, that should return an error to the user. Even if the seed is included somewhere, don’t ever be helpful by failing open. Coerce the behavior you want to see in the world.
If you’re, instead, writing software that expects a semi-expanded private key? Regardless of what the user provides you, you’re doing it wrong and should rethink your life decisions.
A Few Closing Remarks
I wrote this blog post in an airport after a 2 hour layover became a 26 hour layover due to a missed connection. If something is less clear about this than my usual fare, it’s probably because of that.
Note that I used the terms “private key” and “secret key” interchangeably here today. In fact, they’re basically synonyms that can be used interchangeably, generally!
That said, I almost always prefer “secret key”, for a couple of reasons:
- “Privacy is not secrecy,” as Eric Hughes noted in A Cypherpunk’s Manifesto (1993), “A private matter is something one doesn’t want the whole world to know, but a secret matter is something one doesn’t want anybody to know. Privacy is the power to selectively reveal oneself to the world.”
- In most cases, you want secrecy for your key material, not privacy.
- Ironically, the output of a KEM is called a “shared secret” which is better thought of as a “private” key, from Hughes’ delineation.
- “Secret key” can be abbreviated as “SK” while the other component (public key) becomes “PK”, which makes for handy notation.
- Private/Public both start with “P”, which makes this less friendly.
However, other discussions about this topic talked about “private key” formats, and I didn’t want anyone to feel like they missed something. You didn’t!
Many cryptographers have gone on to say “encapsulation key” / “decapsulation key” for KEMs and “signing key” / “verifying key” for signatures to sidestep the “private” vs “secret” confusion. I applaud this, but that doesn’t help when PKCS8 rears its ugly head.
