Loading an X.509 certificate with an empty subject

In October 2026, a report on the mailing list caught my eye: iwd couldn’t connect to an eduroam network because loading the server’s TLS certificate into the kernel keyring kept failing with -EBADMSG. Nobody had replied or picked it up yet, so I started digging in.

At first glance, nothing looked wrong with the certificate. Under RFC 5280, an X.509 certificate is allowed to leave its Subject field completely empty as long as the identity lives in a Subject Alternative Name (SAN) extension. Plenty of modern server certificates do this, and iwd hands them off to the kernel’s asymmetric keyring to parse and verify. To reproduce it myself, I generated two test certificates that differed only in the subject (/ versus CN=radius.example.org). openssl verify was happy with both. The kernel wasn’t: on both Ubuntu’s 6.8 kernel and Fedora 45’s 7.2.15 kernel, the named certificate loaded fine while the empty-subject one came back with “Bad message”.

To see why the parser was bailing out, I built scripts/asn1_compiler and looked at the bytecode table it generates from crypto/asymmetric_keys/x509.asn1. Both issuer and subject shared the same rule: Name ::= SEQUENCE OF RelativeDistinguishedName. The compiler turns a SEQUENCE OF into a loop that expects at least one element. When the subject sequence is empty, there are no bytes left to match, so lib/asn1_decoder.c flags a data overrun and userspace gets -EBADMSG.

The obvious thing to try first was marking the rule OPTIONAL. That compiled without a single warning, and did absolutely nothing. Digging into scripts/asn1_compiler.c, I found that the compiler attaches the skippable flag to the type reference and then quietly drops it when expanding that reference. Fixing the compiler itself would have changed other ASN.1 grammars in the tree, like nf_nat_snmp_basic.asn1, which felt too broad for a bug scoped to X.509.

My next attempt stayed inside x509.asn1 and tagged the first element explicitly so the skippable flag survived compilation. Before sending it out, checking what the generated bytecode did on malformed certificates caught a nasty bug: if a malformed certificate put a non-SET element inside the subject, the decoder would skip it without advancing the input pointer and loop back to the start of the SEQUENCE OF forever. I booted a kernel with that patch and fed it a malformed test cert to see for myself, and sure enough, it hung immediately.

What ended up working was giving the subject its own rule wrapped in a CHOICE:

SubjectName ::= SEQUENCE OF CHOICE {
	rdn	RelativeDistinguishedName
}

Now an empty subject passes through, while anything that isn’t a valid SET hits COND_FAIL and still returns -EBADMSG. Better yet, x509_akid.asn1 and pkcs7.asn1 already use this exact pattern elsewhere in the tree. The issuer field stays on the original Name rule, since RFC 5280 requires a non-empty issuer.

Original SEQUENCE OF MATCH (SET) END_SEQ_OF empty subject → -EBADMSG v1 (reverted) tagged OPTIONAL MATCH_OR_SKIP END_SEQ_OF non-SET skipped, dp unchanged → loops forever malformed input → kernel hang Final CHOICE MATCH_OR_SKIP COND_FAIL END_SEQ_OF empty → OK non-SET → -EBADMSG

One edge case to keep in mind: if you load two empty-subject certificates that have no subject key identifier and share the same serial number, they end up with the same key description and the second replaces the first. Named-subject certificates already work the same way when descriptions collide.

The patch is in the cryptodev tree as de540915632c (lore thread).

Comments