Eight CA Keys Between 1 and 255: What the eMudhra Incident Means for Your Code

Up to date as of 25 October 2026. The fixes are still changing: on 24 October Oracle shipped a second Java fix (CVE-2026-114820), so guidance written before then calls Java builds fixed that aren't. For anything newer, check the incident site's list of patches.

On 17 October 2026 someone signed the sentence "We have a problem here - Wayne" with the private keys of eight intermediate certificate authorities: six trusted for TLS by Chrome, Firefox, Safari and Windows, and two trusted by Windows for code signing. Nobody stole those keys. Nobody had to. Each one was an integer between 1 and 255, so anyone could compute it from the public certificate in less time than it takes to read this sentence.

Chrome and Firefox were protected within hours, Windows and Apple devices within days. Your backend probably wasn't, and the reason says something uncomfortable about TLS outside the browser: trust doesn't live in the protocol, it lives in a file of root certificates that every runtime picks for itself. Patch the OS and you've fixed curl. You haven't fixed Node, a Go binary built last month, or the certifi inside a virtualenv.

One chain runs through the whole article, the real one served by the incident's test host dv-g3b-emudhra.we-have-a-problem.com. We'll look at what a certificate proves, what went wrong with these, what an attacker needs to use them, the before and after on that chain, and how to fix the four runtimes you're most likely to ship: Java, Go, Node.js and Python.

What a Certificate Actually Proves

Definition. A certificate binds a public key to a name, and a signature from someone else backs that binding. That "someone else" is a certificate authority, whose own certificate is signed by another CA, until you reach a root: a certificate nobody signs, which your client trusts because it's in a file called a trust store. The server's own certificate, the one with the hostname in it, is the leaf. The CAs between the leaf and the root are intermediates.

You can print the chain the test host serves:

openssl s_client -connect dv-g3b-emudhra.we-have-a-problem.com:443 -showcerts </dev/null 2>/dev/null | grep -E ' s:| i:'

 0 s:CN=dv-g3b-emudhra.we-have-a-problem.com
   i:C=NL, O=eMudhra BV, CN=EM DV TLS CA - G3B
 1 s:C=NL, O=eMudhra BV, CN=EM DV TLS CA - G3B
   i:C=IN, O=eMudhra Technologies Limited, CN=emSign Root TLS CA - G3
 2 s:C=IN, O=eMudhra Technologies Limited, CN=emSign Root TLS CA - G3
   i:C=IN, OU=emSign PKI, O=eMudhra Technologies Limited, CN=emSign ECC Root CA - G3

s: is who each certificate speaks for and i: is who signed it. The server doesn't send the root; the client has to find it in its own trust store. Drawn as a tree:

dv-g3b-emudhra.we-have-a-problem.com           leaf: "this key speaks for this hostname"
└─ signed by EM DV TLS CA - G3B                intermediate: one of the eight weak keys
   └─ signed by emSign Root TLS CA - G3        intermediate
      └─ signed by emSign ECC Root CA - G3     root: the one in your trust store

During the handshake the server sends the chain, and the client checks three things: every signature verifies with the issuer's public key, every certificate is in date and allowed to do what it's doing, and the chain ends at a root in the client's own trust store. The last check is the only place where trust enters the system. Everything else is arithmetic.

An intermediate's certificate shows how much power it holds. The incident's proof folder has all eight; download one and print its limits:

curl -sSLO https://raw.githubusercontent.com/galadran/we-have-a-problem.com/main/proof/EM-OV-TLS-CA-G3C.crt.pem
openssl x509 -in EM-OV-TLS-CA-G3C.crt.pem -noout -subject -ext basicConstraints,extendedKeyUsage

subject=C=US, O=eMudhra Inc, CN=EM OV TLS CA - G3C
X509v3 Extended Key Usage:
    TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Basic Constraints: critical
    CA:TRUE, pathlen:0

CA:TRUE, pathlen:0 means it may sign leaves but not further CAs. There's no nameConstraints extension, so nothing limits which hostnames it can back. Whoever holds its private key can sign a certificate for api.your-bank.com, and your client has no way to tell it from a real one. That's why CA keys live in hardware security modules (HSMs: tamper-resistant devices that never let the key out) and why key ceremonies have witnesses.

After the fact, two mechanisms can still save a client:

• Revocation. The CA publishes "stop trusting this one" through CRLs (lists of revoked certificates the CA publishes) or OCSP (a service the client asks about one certificate). It works only if the client checks, and most non-browser clients don't by default.
• Certificate Transparency (CT). Public, append-only logs where CAs record every certificate they issue, so anyone can see what was issued for a domain. Browsers reject a leaf that hasn't been logged, so a forged certificate either becomes public or gets rejected. Almost nothing outside the browser checks this.

If you remember one thing: a certificate is only as trustworthy as the private key of every CA above it, and the trust store is a file your client chooses, not a property of the internet.

What Went Wrong at eMudhra

A P-384 private key is a random 384-bit number k. The public key is k·G, the curve's base point added to itself k times. Going forward is cheap; going back is the problem P-384's security rests on. Mozilla's analysis in the incident report found that the eight keys were all integers less than 256: 48 bytes, all zero except the last. With 255 candidates, "going back" means computing 255 points and comparing them with the certificate. One write-up measured 28.7 milliseconds for the full table.

You don't need to brute-force anything to check it. One of the keys is already public: the key of EM OV TLS CA - G3C is 4. Using the certificate you downloaded above:

from cryptography import x509
from cryptography.hazmat.primitives.asymmetric import ec

cert = x509.load_pem_x509_certificate(open("EM-OV-TLS-CA-G3C.crt.pem", "rb").read())
four = ec.derive_private_key(4, ec.SECP384R1())   # the private key "4"

print(cert.public_key().public_numbers() == four.public_key().public_numbers())
# True

P-384 isn't broken. A key of 4 is a valid key: it parses, it signs, signatures verify. Every format check and every sign-and-verify test passes, which is exactly why nothing caught it for almost eighteen months.

How it happened, according to eMudhra's preliminary explanation in the report: the keys were generated on Utimaco HSMs during a ceremony on 30 April 2025. An approved deviation added a test of key wrap/unwrap (exporting a key encrypted and importing it back), a flaw in that test code created a second, weaker key pair inside the HSM, and because the labels were similar the weak pair was used for the CSRs (certificate signing requests: the public key and name a CA is asked to sign). The root signed them and Sectigo cross-signed them. Other participants in the report point out that an HSM's certified key generation doesn't produce a scalar below 256 on its own, so how that material got there is still an open question. eMudhra hasn't answered yet; it says its full report will come within the 14 days the rules allow.

What was affected: eight intermediates from that ceremony.

• Six sign HTTPS certificates: EM DV TLS CA, EM OV TLS CA and EM EV TLS CA, each in a G3B and a G3C version. All six chain to emSign ECC Root CA - G3.
• Two sign code: EM OV CS CA - G3B and EM OV CS CA - G3C. Only Microsoft trusted them.

The names follow a pattern:

• DV, OV and EV say how much the CA checks about a customer before issuing. Domain Validated checks only that they control the domain, Organization Validated also checks that their company exists, and Extended Validation adds stricter legal checks.
• G3B and G3C are two copies of each CA, issued to two eMudhra companies: every G3B to eMudhra BV in the Netherlands, every G3C to eMudhra Inc in the US.

None of it matters for this incident. The checks are a promise the CA makes, not something the key enforces, so whoever holds any of the six keys can sign for any hostname.

Sectigo's AAA Certificate Services root had cross-signed the same eight public keys (signed them too, so they also chain to Sectigo's root), and a cross-sign with the same key is the same compromise. eMudhra itself issued no end-entity certificates (leaves) from them. Whether anyone else did in the eighteen months before the report, nobody can prove either way for clients that don't check CT.

eMudhra has since announced that it is leaving the public TLS business, and some trust stores now drop more than the one root behind these keys. Java's second update removes all three eMudhra roots the JDK carried, and certifi 2026.10.23 drops two more (eMudhra Inc's emSign Root CA - C1 and emSign ECC Root CA - C3). If a server you call still uses an eMudhra certificate, expect it to fail once you update.

The incident site keeps the authoritative list of who has patched.

If you remember one thing: the keys weren't stolen, they were computable, from public data, by anyone, for almost eighteen months. Treat every client that trusted these roots as if the keys had been public all along, because they effectively were.

Why It's Exploitable, and What an Attacker Still Needs

With an intermediate key, an attacker signs a leaf for whatever hostname your service calls. The chain validates. But a certificate alone does nothing: the attacker has to receive your connection to present it. That means a position on the path:

• DNS answers they control (poisoning, a compromised resolver, a hijacked record).
• A network device in the middle: a proxy, a router, a hostile Wi-Fi, a compromised egress gateway.
• A routing attack (BGP hijacking: announcing the destination's IP addresses from a network the attacker controls, so traffic goes there).

This isn't remote code execution, and nobody can scan the internet with it. It turns a network position, which TLS normally makes harmless, into full read-and-write access to the traffic.

So why were browsers safe and your backend not? Because of the two mechanisms above:

• Revocation. eMudhra revoked the intermediates on 18 October. Python, Node, curl and Go on Linux don't check revocation by default, and neither does Java's TLS stack unless you set com.sun.net.ssl.checkRevocation=true. (Go on macOS and Windows asks the OS verifier, so it picked up Apple's and Microsoft's blocks.)
• Certificate Transparency. No certificate signed by these intermediates appeared in the public CT logs before the block. A browser rejects unlogged certificates, so a forgery would have had to show up in public first. Non-browser clients don't check CT, so for them a forged certificate needs no log entry at all.

Browsers also receive blocklists pushed by the vendor (Chrome's CRLSet, Firefox's OneCRL) without a software update. Your runtime gets nothing until you upgrade it.

Forged certificates stopped being hypothetical on 20 October. Over the next day and a half, people logged 107,107 of them, signed with six of the keys, to public CT logs: 106,050 are proof-of-concept certificates for google.com, and every signature verifies. All but two carry no revocation pointer (the CRL or OCSP address a client would ask), so a client that checks revocation only for the leaf, a common setup, finds nothing to check. Only a client that checks every CA in the chain, receives a pushed blocklist or has dropped the roots rejects them.

What that means for a typical service:

Note the second row. "We only call internal services" protects you only if the client trusts only internal CAs. Most services keep the default trust store and append their private CA to it, so a forged certificate for orders.internal still validates. It's just much harder to deliver.

If you remember one thing: the key is the weapon and the network position is the trigger. Your risk is how many networks you don't control sit between your clients and the servers they call.

Before and After

The incident site runs test hosts that serve the compromised chains. Run the check from where your code runs, a pod, a container or a CI job, not from your laptop:

curl -sS -o /dev/null -w '%{http_code}\n' https://dv-g3b-emudhra.we-have-a-problem.com/      # eMudhra root
curl -sS -o /dev/null -w '%{http_code}\n' https://dv-g3b-aaa-emudhra.we-have-a-problem.com/  # Sectigo cross-sign

# Before: the client trusts the compromised intermediate
200

# After: trust removed (wording depends on curl's TLS backend; this is OpenSSL)
curl: (60) SSL certificate problem: unable to get local issuer certificate

Two more hosts serve leaves that were never logged to CT:

curl -sS -o /dev/null -w '%{http_code}\n' https://dv-g3b-noct-emudhra.we-have-a-problem.com/
curl -sS -o /dev/null -w '%{http_code}\n' https://dv-g3b-aaa-noct-emudhra.we-have-a-problem.com/

A 200 here is the worst result: your client trusts the compromised intermediates and doesn't enforce CT, so it could be attacked without anything ever appearing in a public log.

curl only tests curl's trust store. Your service uses its runtime's, so the companion script runs the same request through curl, Python, Node, Java and Go, whichever are installed.

Before you run it at work. Each check opens one HTTPS connection to a test host. The incident site runs them; Oracle and OpenJDK link to it, and its source is on GitHub. The hosts see what any website sees: your public IP, the hostname and the time. curl, Python, Node and Go send a plain GET /, and the Java probe stops after the TLS handshake. A host your client trusts answers with one line of text, which the scripts throw away. On a company network, three things matter:

• A connection that presents a certificate from a revoked, compromised CA is exactly what a security proxy or intrusion detection system should flag. Tell your security team before you run it.
• If a proxy inspects TLS, your client checks the proxy's certificate instead of the test host's, so the result tells you nothing about your runtime.
• Proxies often block newly registered domains. That shows up as ERROR, which proves nothing either way.

If you can't make the call, the offline check below and Probe --roots in the Java section need no connection to the test hosts.

Offline, with no network at all. The site also publishes every chain as PEM (the text format that starts with -----BEGIN CERTIFICATE-----), so you can check a trust store file directly. Here is the same chain against certifi before and after the fix:

$ openssl verify -CAfile certifi-2026.7.22.pem -untrusted chain.pem chain.pem
chain.pem: OK

$ openssl verify -CAfile certifi-2026.10.19.pem -untrusted chain.pem chain.pem
C = IN, O = eMudhra Technologies Limited, CN = emSign Root TLS CA - G3
error 20 at 2 depth lookup: unable to get local issuer certificate
error chain.pem: verification failed

OK means the bundle still trusts the compromised intermediate. Error 20 at depth 2 means the chain now dead-ends one step before the root: emSign ECC Root CA - G3 is gone from the bundle. The Sectigo chain (dv-g3b-aaa) fails against both versions, because certifi doesn't carry AAA Certificate Services at all. That cross-sign path matters for stores that do: Windows, the /etc/ssl/cert.pem macOS ships, and the Linux stores the incident site gives workarounds for.

If you remember one thing: test with the runtime your service uses, from where it runs. A green curl on your laptop says nothing about a Node container in production.

Why Every Runtime Needs Its Own Fix

Each runtime answers "which roots do I trust?" differently, so each one got a different fix:

Go's fix is a list of the eight compromised public keys (their SHA-256 hashes) in crypto/x509, and Verify rejects any chain that contains one of them. Blocking by public key rather than by certificate is the right call: the eMudhra certificate and the Sectigo cross-sign carry the same key, so one entry covers both. And because the list is part of the standard library, it ends up inside every binary you build. Upgrading the Go installed on a server changes nothing for a binary that's already compiled: you have to rebuild it with 1.26.10 or 1.27.3.

Two consequences are easy to miss:

• "Update the OS" is not a fix for Node or certifi. Neither reads the OS store by default. For Go on Linux it helps, because it drops the root, but the blocklist only arrives with a rebuild.
• Your CVE scanner won't flag Node or Python. Neither fix has a CVE, so a scanner that matches CVEs to versions reports them clean. Check those two by version. For Java, make sure the scanner knows CVE-2026-114820: against the first CVE alone, the 21 October builds look fixed.

If you remember one thing: ask of every process, "where does this get its roots, and who updates that file?" The answer is different for each runtime, and sometimes it's "a file compiled into a binary you built last month."

Fixing It, Runtime by Runtime

Java

Oracle shipped two fixes three days apart, and only the second one is enough. Both score 9.1 and cover Java SE 8, 11, 17, 21, 25 and 27, GraalVM for JDK 17 and 21, and GraalVM Enterprise Edition 21.3.

• CVE-2026-91403, 21 October: distrusts the eight intermediates and their eight Sectigo cross-signs through blocked.certs. Oracle's builds 11.0.33.0.2, 17.0.21.0.2, 21.0.13.0.2, 25.0.5.0.2, 27.0.1.0.2 and 8u511 b32.
• CVE-2026-114820, 24 October: also removes four roots from cacerts: emSign ECC Root CA - G3, emSign Root CA - G1, emSign Root CA - G2 and Sectigo's AAA Certificate Services. It lists the 21 October builds as affected. Oracle's fixed builds are 11.0.33.0.3, 17.0.21.0.3, 21.0.13.0.3, 25.0.5.0.3, 27.0.1.0.3 and 8u511 b33. Java 8 keeps the update number and bumps the build, so java -version has to say 1.8.0_511-b33.

Only the first and the last of those roots are behind the compromised chains; the JDK drops eMudhra's other two as well.

• Update the JDK in every place it lives: Docker base images, setup-java in CI, self-hosted runners, laptops. For Docker, rebuild your own image with docker build --pull so it starts from the updated base. Redeploying an image you already built, or giving it a new tag, keeps the old JDK inside it. Other vendors (Temurin, Corretto, Zulu and the rest) number their builds differently. OpenJDK's eMudhra page, last updated on 24 October, says to look for JDK-8395690 / CVE-2026-114820 in your vendor's release notes, "even if you have updated your runtime in the past few days". It also publishes CheckCerts.java, which prints BAD ROOT for each of the four roots a runtime still trusts.
• Base images lag behind vendor releases. On 25 October, three days after Temurin's October builds came out, the official eclipse-temurin:21-jre, 21-jre-alpine and 25-jre-alpine images still shipped 21.0.12.1 and 25.0.4.1, and Probe (below) still saw them trust the test hosts. Until the image you build on is updated, --pull changes nothing: check the result, don't assume it.
• If your vendor hasn't shipped the fix yet, remove the four roots yourself, then restart the JVM. OpenJDK's page deletes them by alias, but alias names differ between vendors (Temurin calls the first one cn_emsign_ecc_root_ca__g3,o_emudhra_technologies_limited,ou_emsign_pki,c_in [jdk]), so find them by subject instead:

  keytool -list -v -cacerts -storepass changeit \
    | awk '/^Alias name: /{a=substr($0,13)} /^Owner: .*CN=(emSign Root CA - G[12]|emSign ECC Root CA - G3|AAA Certificate Services)(,|$)/{print a}' \
    | while IFS= read -r a; do keytool -delete -cacerts -storepass changeit -alias "$a"; done

It prints nothing when it works, and nothing when there's nothing to delete. To see what it will remove, run the first two lines on their own: they print the aliases that match. On eclipse-temurin:21-jre:

  cn_emsign_ecc_root_ca__g3,o_emudhra_technologies_limited,ou_emsign_pki,c_in [jdk]
  cn_emsign_root_ca__g1,o_emudhra_technologies_limited,ou_emsign_pki,c_in [jdk]

Run them again afterwards: no output means the roots are gone. Deleting needs write access to cacerts, so a system JDK needs sudo; without it, keytool stops with Permission denied.

changeit is the default password of the JDK's cacerts. Java 8's keytool has no -cacerts option, so replace it with -keystore and the file: $JAVA_HOME/jre/lib/security/cacerts in a JDK, $JAVA_HOME/lib/security/cacerts in a JRE. A vendor's cacerts doesn't always hold all four (Temurin's has emSign Root CA - G1 and emSign ECC Root CA - G3 only), and the command deletes whichever it finds. Whether you update or remove them by hand, servers that chain to AAA Certificate Services or to eMudhra's other roots stop working, so test the hosts you call before rolling it out.

• In a Docker image, the keystore is baked into the image, so the removal goes in your Dockerfile, after the FROM:

  FROM eclipse-temurin:21-jre
  RUN keytool -list -v -cacerts -storepass changeit \
      | awk '/^Alias name: /{a=substr($0,13)} /^Owner: .*CN=(emSign Root CA - G[12]|emSign ECC Root CA - G3|AAA Certificate Services)(,|$)/{print a}' \
      | while IFS= read -r a; do keytool -delete -cacerts -storepass changeit -alias "$a"; done

To check an image on your machine, ask the Java inside it. The companion's Probe.java connects to the test hosts through the JVM's own TLS stack, so it sees what your application sees: the image's cacerts, the blocked.certs fix and any trust store the image configures. The version string can't tell you that, least of all for a company's own JDK build. Compile it once with javac --release 8 Probe.java, then run it with the image's Java (8 or later; a JRE is enough):

  docker run --rm --entrypoint java -v "$PWD:/probe:ro" my-image -cp /probe Probe

Any TRUSTED line means the image is still exposed. --entrypoint makes this work even when the image starts your application by default, and settings in its ENV, such as JAVA_TOOL_OPTIONS, still apply. If the start script passes -Djavax.net.ssl.trustStore=… instead, add the same flags after my-image. On your laptop, java -cp . Probe does the same for your own JDK.

• Company images: only your CAs, or the public roots too? The answer tells you which row of the exposure table above you're in. Probe --roots lists every root the image's Java trusts, and it needs no network:

  docker run --rm --entrypoint java -v "$PWD:/probe:ro" my-image -cp /probe Probe --roots

  /opt/java/openjdk 21.0.12.1+1-LTS, trustStore=/opt/corp/truststore.p12
       CN=ACME Corp Root CA,O=ACME
  1 trusted roots: 0 BAD (behind the compromised chains), 0 DROP (also removed by the 24 October fix)

That image trusts one private root and nothing else. The stock eclipse-temurin:21-jre lists 118 roots, including:

  BAD  CN=emSign ECC Root CA - G3,O=eMudhra Technologies Limited,OU=emSign PKI,C=IN
  DROP CN=emSign Root CA - G1,O=eMudhra Technologies Limited,OU=emSign PKI,C=IN

BAD marks a root behind the compromised chains. DROP marks eMudhra's other two roots: they aren't compromised, but the 24 October fix removes them too, and OpenJDK's CheckCerts flags them as well. Either label makes Probe exit with 1. On a fixed JDK no line is labelled, the summary reads 0 BAD …, 0 DROP …, and Probe exits with 0; the number of roots varies by vendor and doesn't matter. Ask Java rather than reading the trust store file with keytool: a PKCS12 file (the standard keystore format, Java's default since 9) protected by a password lists no entries without that password, so it looks empty when it isn't. Java gets the password from the image's own settings.

This covers Java only. curl, git, Python's ssl module and Go read the OS bundle (/etc/ssl/certs/ca-certificates.crt on Debian, Ubuntu and Alpine), and company images often add their CA to it rather than replace it. docker create lets you copy it out without starting anything:

  id=$(docker create my-image) && docker cp -L "$id":/etc/ssl/certs/ca-certificates.crt . ; docker rm "$id"
  openssl crl2pkcs7 -nocrl -certfile ca-certificates.crt | openssl pkcs7 -print_certs -noout | grep '^subject'

• Know which cacerts you use. readlink -f "$JAVA_HOME/lib/security/cacerts" tells you. JDK packages on Fedora, RHEL, Arch, Gentoo and Oracle Linux read the OS store, so updating ca-certificates is likely enough there. Homebrew's and FreeBSD's OpenJDK carry their own cacerts, so upgrade the JDK package itself: brew upgrade openjdk, not brew update, which only refreshes Homebrew's list of packages. On 25 October only Homebrew's openjdk (27.0.1.0.3) had the 24 October fix. openjdk@21 (21.0.13.0.2) still trusted all four roots, and the other versioned packages were at the same or older builds, so remove the roots by hand there. On Debian and Ubuntu, cacerts points to /etc/ssl/certs/java/cacerts, and a long-standing bug means updating ca-certificates doesn't reach it. Regenerate it as root:

  rm -f /etc/ssl/certs/java/cacerts
  update-ca-certificates --fresh

• Embedded JREs. Jenkins, Kafka, Elasticsearch, IDEs and many vendor appliances ship their own Java. Updating the system JDK doesn't touch them, so update the product.
• Revocation checking instead of removing the roots. OpenJDK and Oracle both accept -Dcom.sun.net.ssl.checkRevocation=true -Dcom.sun.security.enableCRLDP=true as the alternative. Java checks every certificate in the chain, not just the leaf, so it also rejects the forged certificates that carry no revocation pointer: with those two flags after my-image, Probe rejects all four test hosts even on an unpatched eclipse-temurin:21-jre. It costs you a CRL fetch on new chains and can fail handshakes when a CRL endpoint is unreachable. eMudhra's CRL and OCSP servers have also been refusing curl's, Wget's and Python's User-Agents (Java's got through on 25 October), so test it before rolling it out. For signed JARs, JDK 17 and later can check revocation with jarsigner -verify -revcheck -strict app.jar.
• If you can't update yet, Oracle also publishes a replacement blocked.certs for lib/security/ (on Java 8 the file is blacklisted.certs). It covers only the first CVE, so remove the roots as well.

Go

• See what a binary was built with, not what's installed: go version ./bin/my-service prints the toolchain embedded in the file.
• Rebuild with 1.26.10 or 1.27.3 and redeploy. Pin it where your build reads it: a toolchain go1.26.10 line in go.mod, go-version in setup-go, the golang: builder image. Go 1.25 and older get no fix.
• Rebuild the runtime image too. On Linux, Go still reads the OS store, so a distroless or alpine base from last month keeps the old roots until you pull a new one.

Node.js

• Check what your node trusts, from the container that runs it:

  node -e "const {X509Certificate}=require('crypto');
    const hit=require('tls').rootCertificates.some(p=>new X509Certificate(p).subject.includes('emSign ECC Root CA - G3'));
    console.log(process.version, hit ? 'STILL TRUSTS emSign ECC Root CA - G3' : 'root removed')"
  # v22.22.0 STILL TRUSTS emSign ECC Root CA - G3

• Upgrade to 22.23.4, 24.22.1 or 26.12.1 everywhere you pin a version: .nvmrc, engines, setup-node, the node: base image, Lambda runtimes. These releases also fix other CVEs, including a Brotli out-of-bounds write (CVE-2026-111497), so they're worth taking anyway.
• Node 20 and older get nothing. Node 20 reached end of life in April 2026.
• Mind the flags. NODE_EXTRA_CA_CERTS only adds roots, it never removes one. --use-system-ca and --use-openssl-ca switch Node to the OS store, which makes the OS package part of your fix.
• Electron, Bun and Deno bundle their own roots and released their own fixes.

Python

• Check every environment, not the interpreter: python -c "import certifi; print(certifi.__version__)" inside each virtualenv, image and lambda.
• Pin certifi>=2026.10.19 and regenerate your lockfile so the pin actually lands. Upgrade pip to 26.2.2 or later and uv to 0.13.5 or later. uv ships its own roots, so it needed its own fix: 0.13.4 removed the root and 0.13.5 also rejects chains that carry the compromised keys.
• New Python releases don't fix old virtualenvs. 3.15.1, 3.14.9 and 3.13.17 bundle a newer pip for new installs. Existing environments keep whatever certifi they had.
• Look for vendored bundles. Some libraries ship a .pem inside the package and ignore both the OS and certifi: find .venv -path '*site-packages*' -name '*.pem' lists them.
• Code that uses ssl or urllib directly follows the OS store, so it's fixed by ca-certificates.

The OS and Your Images

• Update ca-certificates on hosts and in images. Patched versions include Debian 20250419~deb12u2 / 20250419+deb13u1, Ubuntu 20260816ubuntu0.24.04.1 (and its 22.04, 26.04 and 26.10 equivalents), Alpine 20261020-r0 and Fedora 2026.2.92_v9.0.319.
• Rebuild, don't just redeploy. An image keeps the trust store it was built with. Images pinned by digest or built from a cached layer never pick up the fix. docker build --pull and check: docker run --rm my-image dpkg -l ca-certificates (or apk info -v ca-certificates on Alpine).
• macOS has no ca-certificates package. Safari, Mail and apps built on Apple's frameworks already block these certificates: Apple pushes that change to devices without an OS update. The command-line tools that ship with macOS (curl, git, python3, ruby, perl, openssl) read /etc/ssl/cert.pem instead, which keeps the root until a future macOS update. Apple's note and the incident site give the commands to remove it yourself; a macOS update may put it back. The built-in pip3 uses its own, older copy of the roots, and the incident site shows how to point it at the fixed file. Tools installed with Homebrew use Homebrew's own bundle: brew upgrade ca-certificates.

CI/CD and Your Dependencies

The CI risk is a forged registry serving a tampered package. TLS was the only thing standing between your build and that package, unless something else checks the content:

A job that resolves dependencies from scratch, without a lockfile, trusts TLS and nothing else.

Managed Runtimes

On AWS Lambda, a function on a managed runtime in Auto update mode gets the patched runtime once AWS publishes one, rolled out in phases. Function update mode only updates when you deploy the function, container images need a rebuild, and anything still on nodejs20.x won't be patched at all. The INIT_START line in the function's logs shows the runtime version it started with.

The Checklist

• Inventory every runtime and version per service: Dockerfiles, CI workflows, go.mod, .nvmrc, requirements*.txt, lockfiles, Lambda configs.
• Java: JDK at a 24 October build (release notes list JDK-8395690) in images, CI, runners, laptops and embedded JREs, or the four roots removed (or revocation checking on), and Probe rejecting all four test hosts.
• Go: every deployed binary rebuilt with 1.26.10 or 1.27.3 (go version ./binary).
• Node.js: 22.23.4, 24.22.1 or 26.12.1 everywhere; nothing left on Node 20.
• Python: certifi>=2026.10.19 in every environment, pip ≥ 26.2.2, uv ≥ 0.13.5, lockfiles regenerated, vendored .pem files reviewed.
• OS: ca-certificates updated on hosts and in base images; images rebuilt with --pull; on macOS, /etc/ssl/cert.pem fixed by hand until Apple's update.
• Custom trust stores: any -Djavax.net.ssl.trustStore, REQUESTS_CA_BUNDLE, SSL_CERT_FILE or bundled .pem checked for emSign ECC Root CA - G3.
• Third-party software: agents, appliances, backup tools and anything else that makes its own TLS connections, checked against its vendor's notice.
• CI: dependency verification on (lockfiles with hashes, Gradle verification metadata).
• Verify from where the code runs: the test hosts reject, including the noct ones.
• Don't stop at the scanner: Node and Python have no CVE for this.

Trust Is a File You Ship

Here's the whole article as two questions to ask of any process that opens a TLS connection:

1. "Where does this get its roots?" The OS store, a bundle inside the runtime, a package in the virtualenv, a list compiled into the binary. If the answer is "I don't know", that's the finding.
2. "Who updates that file, and when do I pick it up?" Your distro, your runtime's release, your lockfile, your next build. Anything that answers "the next build" is only fixed once you build.

None of this is new. DigiNotar in 2011 and the Symantec distrust in 2018 taught the same lesson, and browsers answered it with pushed blocklists and Certificate Transparency. Most server-side clients got neither. Until they do, the trust store is a dependency like any other: it has versions, it goes stale, and it ships inside your artifacts whether you think about it or not.

The check script lives in the companion folder: it sends the same request to the four test hosts through curl, Python (ssl and certifi), Node, Java and Go, whichever are installed, and prints which ones still trust the compromised chains. The chains and the proof certificates used above come from the incident site's repository.