Dev.to Security 🔐 Cybersecurity 👁 0 📖 7 min read

Two liboqs builds in one container, and two GCC 14 bugs

pqc-probe compares a stock client with an experimental post-quantum one, over both TLS and SSH. Each experimental client is built on liboqs, the Open Quantum Safe library, and building them by hand takes an afternoon of

Two liboqs builds in one container, and two GCC 14 bugs

pqc-probe compares a stock client with an experimental post-quantum one, over both TLS and SSH. Each experimental client is built on liboqs, the Open Quantum Safe library, and building them by hand takes an afternoon of cmake, autoconf and environment variables. So the release ships a Docker image with all four clients ready to use.

Getting there took two upstream bug reports, one library-name collision that would have failed silently, and a build that refuses to finish if the result is wrong. This post walks through each.

Part 5 of a series on building pqc-probe. Earlier: part 1, part 2, part 3, part 4.

Four stages, one runtime

The Dockerfile is a multi-stage build. Three builder stages carry the compilers, headers and source trees; the final stage copies only what they produced.

Three builder stages feed one small runtime image

The Rust stage is unremarkable. The two OQS stages are where the work is, and they exist separately for a reason that matters later: the TLS stack and the SSH stack each want their own liboqs. oqs-provider is built against a released liboqs. OQS-OpenSSH's build script clones liboqs itself, from a branch rather than a release. So the runtime image ends up holding two builds of the same library, and the loader has to pick the right one for each client.

The runtime stage adds Debian's own openssl and openssh-client as the stock clients, so the vanilla side of every comparison is exactly what a Debian 13 user has.

Picking versions: the newest pair is not the safest pair

The obvious question is why the image doesn't use the newest liboqs (0.16) and the newest oqs-provider (0.12). The answer is that the two are released as a pair, and the pair that had actually been released together was 0.15.0 and 0.11.0:

  • oqs-provider 0.11.0's release notes say it is in sync with liboqs 0.15.0.
  • oqs-provider 0.12 was still a release candidate when I built the image.
  • liboqs 0.16.0 renames the FrodoKEM algorithms and removes SPHINCS+. A provider written against 0.15 and run against 0.16 is a mismatch waiting to surface as a missing algorithm, which is exactly the kind of quiet failure a scanner can't afford.

OQS-OpenSSH is harder to pin because it has no release tags. The image clones the OQS-v10 branch and checks out a fixed commit (6717c038a). Its build script, oqs-scripts/clone_liboqs.sh, then clones liboqs from main by default, so the SSH side gets whatever liboqs main held that day. In my build that was 9f027e7c, which already calls itself 0.16.0. I didn't fight that: the SSH client is built and tested against the liboqs its own project tracks, and the commit is recorded (more on that below).

Every one of these is a build argument, so trying a newer pair doesn't need an edited Dockerfile:

docker build \
  --build-arg LIBOQS_VERSION=0.16.0 \
  --build-arg OQS_PROVIDER_VERSION=0.12.0 \
  --build-arg OQS_OPENSSH_COMMIT=<commit> \
  -t pqc-probe:try .

If the pair doesn't work, the build fails rather than producing an image that looks fine. That's the job of the self-checks in the last stage.

Two bugs that GCC 14 won't forgive

The first full build died in the oqs-ssh stage, on ssh-oqs.c line 162:

error: implicit declaration of function 'sshkey_clear_pkey'

sshkey_clear_pkey exists. It's declared in sshkey.h at line 367, but inside an #ifdef SSHKEY_INTERNAL block, and ssh-oqs.c does define SSHKEY_INTERNAL. The problem is the order:

#include "oqs-utils.h"     /* line 32 */
#include "sshbuf.h"
#define SSHKEY_INTERNAL      /* line 34 */
#include "sshkey.h"

oqs-utils.h includes sshkey.h itself (its line 5). By the time ssh-oqs.c defines the macro and includes sshkey.h again, the include guard is already set, so the internal declarations are never seen. The fix is to define the macro before the first include:

+#define SSHKEY_INTERNAL
 #include "oqs-utils.h"
 #include "sshbuf.h"
-#define SSHKEY_INTERNAL
 #include "sshkey.h"

With that out of the way, the second one appeared in kexgen.c at line 479: a call to kex_reset_keys. The function is defined in kex.c (line 727), but kex.h has no prototype for it. It came in with commit ca3025ebf. The fix is one line in kex.h, next to its sibling:

 int     kex_derive_keys(struct ssh *, u_char *, u_int, const struct sshbuf *);
+void   kex_reset_keys(struct kex *);

Both are the same class of bug: calling a function the compiler hasn't seen a declaration for. GCC 13 accepts that with a warning, which is why it slipped through. GCC 14 made implicit function declarations an error by default, and Debian 13 ships GCC 14.2. Any distribution on GCC 14 or newer hits both.

For the upstream reports I wanted reproductions that don't need my Dockerfile. In a plain debian:trixie-slim container, after configuring OQS-OpenSSH (at the then-latest commit, 5d42e44d7) against its liboqs, compiling one object is enough to show each failure:

make ssh-oqs.o   # sshkey_clear_pkey
make kexgen.o    # kex_reset_keys

They're filed as open-quantum-safe/openssh#210 and #211. Neither had a reply when I last checked.

Until they're fixed upstream, the Dockerfile patches the source, but only while the bug is still there. The ssh-oqs.c patch runs only if the oqs-utils.h include still comes before the define, and the kex.h patch only if kex.h still lacks the prototype. When upstream fixes either one, its patch becomes a no-op instead of a conflict. The configure step also passes -Werror=implicit-function-declaration, so the build is just as strict on an older compiler.

Two libraries, one name: liboqs.so.9

This is the problem that would have failed silently.

liboqs 0.15.0 installs liboqs.so.0.15.0. liboqs main installs liboqs.so.0.16.0. Both carry the same so name, liboqs.so.9, and the so name is what an executable asks the dynamic loader for at startup. The runtime image has both: the TLS stack's copy in /opt/liboqs/lib, registered with ldconfig so OpenSSL can load the provider, and the SSH stack's copy next to the OQS ssh binary.

Without further help, the loader resolves liboqs.so.9 for ssh through the system library cache, which points at the 0.15.0 build. The SSH client would then run against a liboqs it wasn't compiled for. At best that fails at startup on a missing symbol. At worst it starts and offers a different set of algorithms than the build intended, and the scanner reports it as fact.

The fix is a run path. The SSH client is linked with -Wl,-rpath,/opt/oqs-openssh/lib, and its liboqs is copied into that directory. Debian's linker records this as RUNPATH, which the loader searches before the system cache. Two commands confirm it inside the image:

readelf -d /opt/oqs-openssh/bin/ssh | grep RUNPATH
#  (RUNPATH)  Library runpath: [/opt/oqs-openssh/lib]

ldd /opt/oqs-openssh/bin/ssh | grep liboqs
#  liboqs.so.9 => /opt/oqs-openssh/lib/liboqs.so.9 (...)

The general lesson: when an image holds two builds of one library, check their so names. Two different version numbers on the files doesn't mean two different names to the loader.

A build that checks itself

A scanner image that builds but quietly lacks its post-quantum algorithms is worse than one that fails to build. So the runtime stage ends with three checks, and any one failing stops the build:

openssl-oqs list -providers      | grep -qi oqsprovider
openssl-oqs list -kem-algorithms | grep -qi frodo
/opt/oqs-openssh/bin/ssh -Q kex  | grep -q openquantumsafe

The first proves the provider loads. The second proves it exposes algorithms that only liboqs provides: FrodoKEM isn't in stock OpenSSL, so seeing it means the provider is really talking to liboqs (my build lists 16 FrodoKEM variants). The third proves the OQS ssh starts with the right liboqs and offers the @openquantumsafe.org key exchanges (17 of them in my build).

The last step writes /opt/pqc-probe/VERSIONS, so anyone can see exactly what an image contains without rebuilding it:

docker run --rm --entrypoint cat ghcr.io/asifmahbubnaeem/pqc-probe:0.1.0 /opt/pqc-probe/VERSIONS
liboqs (TLS): 0.15.0
oqs-provider: 0.11.0
OQS-OpenSSH: OQS-v10 @ 6717c038…
liboqs (SSH): 9f027e7c…
stock openssl 3.5.7
stock ssh OpenSSH_10.0p2 Debian-7+deb13u4
experimental ssh OpenSSH_10.4-2026-08_p1, Open Quantum Safe 2026-08

The liboqs line for SSH is a commit, not a version, because that's the only honest description of a build from main. When a probe result looks odd, this file is the first thing to attach to a bug report.

Takeaways

  • Pin, and record what you pinned. Releases where they exist, commits where they don't, and write both into the image.
  • Prefer pairs that were released together. For liboqs and oqs-provider, the newest of each isn't a tested pair.
  • Make the build verify its own output. A failed build is cheap; an image that silently lacks its algorithms is not.
  • Check so names when you ship two builds of one library. Different file versions can share one name, and the loader only sees the name.
  • Fix it locally, report it upstream. Patches that apply only while the bug exists turn into no-ops when upstream catches up.

The image is on GHCR as ghcr.io/asifmahbubnaeem/pqc-probe, and the Dockerfile is in the repo. Next in the series: running a scanner that strangers can point at any address, and what it takes to keep it from being turned against you.

📰 Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.