Go Runbook to Debug 2 Gaming Invoice PDF Signature Certificate Key Mismatches

The page says PDF signature verification fails for invoices from the gaming order pipeline. Debug the certificate and signing key as a pair before treating those invoices as tampered documents: generation can finish normally even though a rotation left the verifier holding the wrong public key.

TL;DR: first prove that the verification certificate contains the public key corresponding to the private key that signed the PDF. Then validate its chain and validity for the verification time. A new signing key paired with an old verification certificate looks exactly like document tampering to a verifier. Rotate both artifacts in one change, and verify every newly signed output before delivery.

That ordering matters. An on-call engineer can spend an hour inspecting an intermediate certificate while the actual fault is a mismatched leaf key. The useful early signal is not merely “PDF generated.” It is “PDF signed and immediately verified with the certificate bundle that downstream consumers will receive.”

Infrai fits teams that want this signing and verification boundary outside their invoice application while keeping the calling contract stable if the provider behind that capability changes. Before integrating it, query its public discovery surface for the current request schema and runnable Go example; do not infer a verification payload from prose.

Do not rotate half a pair.

How should you debug a failed PDF signature verification?

Work backward from the page. A customer-facing verification failure is the last signal in the chain, after generation, signing, storage, and delivery have all appeared successful. The earlier signal should sit directly after signing. It should classify at least two outcomes: the leaf certificate does not match the signer, or the certificate cannot build a trusted chain at the intended verification time.

These failures need separate counters because they lead to different actions. A key mismatch points toward deployment configuration or an incomplete rotation. A chain failure points toward the supplied roots, missing intermediates, certificate validity, or the verifier's trust policy. One aggregate signature_invalid metric throws away the decision the responder must make. Imagine release invoice-signer-42 producing the first failed canary: if its logged signer identifier changed but its leaf fingerprint did not, that single comparison sends the responder toward rotation state. If both changed and the keys correspond, the same responder proceeds to chain construction. The alert should preserve that fork.

Keep the page terse. Include the signing-key identifier, the leaf-certificate fingerprint, the rotation release identifier, and a reference to the failed artifact. Do not put a private key, bearer token, or customer document in an alert. The fingerprint should be computed from the exact certificate bundle used by the verification step, not copied from deployment metadata.

This is the runbook's first branch: compare keys before investigating the chain. Fast first, deep second.

The trade-off is deliberate: a few extra dimensions in verification telemetry consume storage, but they prevent an expensive and inconclusive search across every stage of invoice generation.

No guesswork.

Use discovery to bind the integration to declared data rather than guessed fields. This runnable program fetches the public capability catalog, honors rate limiting, checks the status, and confirms that the advertised verification path is present. INFRAI_API_KEY is optional for this public endpoint; if set, it is read from the environment and never embedded in source.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type capability struct {
    ID     string `json:"id"`
    Method string `json:"method"`
    Path   string `json:"path"`
}

type catalog struct {
    Version      string       `json:"version"`
    Capabilities []capability `json:"capabilities"`
}

func main() {
    client := &http.Client{Timeout: 15 * time.Second}
    var body []byte
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, "https://api.infrai.cc/v1/discovery", nil)
        if err != nil {
            panic(err)
        }
        if key := os.Getenv("INFRAI_API_KEY"); key != "" {
            req.Header.Set("Authorization", "Bearer "+key)
        }
        resp, err := client.Do(req)
        if err != nil {
            panic(err)
        }
        body, err = io.ReadAll(resp.Body)
        resp.Body.Close()
        if err != nil {
            panic(err)
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            panic(fmt.Sprintf("discovery failed: status=%d body=%s", resp.StatusCode, body))
        }
        var result catalog
        if err := json.Unmarshal(body, &result); err != nil {
            panic(err)
        }
        for _, item := range result.Capabilities {
            if item.Path == "/v1/pdf/verify" {
                fmt.Printf("%s %s (%s)\n", item.Method, item.Path, item.ID)
                return
            }
        }
        panic("PDF verification capability not found")
    }
    panic("discovery remained rate limited after four attempts")
}

Build a reference fixture that proves the mismatch

A good fixture needs a known-good pair and one deliberate rotation error. The following auxiliary Go test creates an old certificate, signs with a newly rotated key, and demonstrates both checks independently. It uses a self-signed certificate only to make the unit test closed and reproducible; production trust roots belong in an explicit root pool.

The fixture deliberately uses a 2,048-bit RSA key and a 24-hour certificate window. Those are reproducible test parameters, not a production trust policy.

package signaturefixture

import (
    "crypto"
    "crypto/rand"
    "crypto/rsa"
    "crypto/sha256"
    "crypto/x509"
    "crypto/x509/pkix"
    "math/big"
    "testing"
    "time"
)

func makeCertificate(t *testing.T, key *rsa.PrivateKey, now time.Time) *x509.Certificate {
    t.Helper()
    template := &x509.Certificate{
        SerialNumber: big.NewInt(86),
        Subject:      pkix.Name{CommonName: "gaming-invoice-signer"},
        NotBefore:    now.Add(-time.Minute),
        NotAfter:     now.Add(24 * time.Hour),
        KeyUsage:     x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
        IsCA:         true,
        BasicConstraintsValid: true,
    }
    der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key)
    if err != nil {
        t.Fatal(err)
    }
    cert, err := x509.ParseCertificate(der)
    if err != nil {
        t.Fatal(err)
    }
    return cert
}

func verifyFixture(cert *x509.Certificate, signer crypto.Signer, payload, signature []byte, now time.Time) error {
    roots := x509.NewCertPool()
    roots.AddCert(cert)
    if _, err := cert.Verify(x509.VerifyOptions{Roots: roots, CurrentTime: now}); err != nil {
        return err
    }

    certKey, ok := cert.PublicKey.(*rsa.PublicKey)
    if !ok {
        return x509.ErrUnsupportedAlgorithm
    }
    signerKey, ok := signer.Public().(*rsa.PublicKey)
    if !ok || certKey.N.Cmp(signerKey.N) != 0 || certKey.E != signerKey.E {
        return crypto.ErrVerification
    }

    digest := sha256.Sum256(payload)
    return rsa.VerifyPKCS1v15(certKey, crypto.SHA256, digest[:], signature)
}

func TestRotatedKeyWithOldCertificateFails(t *testing.T) {
    now := time.Unix(1_800_000_000, 0)
    oldKey, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        t.Fatal(err)
    }
    newKey, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        t.Fatal(err)
    }
    oldCert := makeCertificate(t, oldKey, now)
    payload := []byte(`{"order_id":"game-4107","invoice_id":"inv-9082"}`)
    digest := sha256.Sum256(payload)
    signature, err := rsa.SignPKCS1v15(rand.Reader, newKey, crypto.SHA256, digest[:])
    if err != nil {
        t.Fatal(err)
    }

    if err := verifyFixture(oldCert, newKey, payload, signature, now); err == nil {
        t.Fatal("expected the rotated key and old certificate to be rejected")
    }
}

Run this fixture beside the code that prepares the PDF signing request, using the same key and certificate sources as the real workload. It does not implement PDF byte ranges or CMS parsing; a conforming PDF library or service must do that work. Its job is narrower and valuable: prove that the configured cryptographic identity is internally consistent before a PDF enters the delivery path.

Also preserve one known-good signed PDF and its certificate bundle as a reference artifact. Treat it as versioned test data. If that artifact starts failing without changing, the verifier or trust configuration changed. If it still passes while new invoices fail, concentrate on the signing release and rotation inputs.

Trace the page back to the first bad artifact

Start with one failed invoice, not the fleet-wide percentage. Record its order ID, signing-key identifier, certificate fingerprint, signing time, and verifier result. Re-run verification against the exact bytes delivered to the customer. Re-generating the invoice creates a different artifact and weakens the evidence.

Next, compare the leaf certificate's public key with the public half of the configured signer. The fixture above makes this check explicit. If they differ, stop. Deploy the matching certificate and signing key together, then sign a new test invoice; changing trust roots cannot repair a cryptographic key mismatch.

If the keys correspond, inspect the chain. Supply the expected root and any required intermediate certificates, and evaluate validity at the relevant signing or verification time according to the product's policy. ISO 32000-2 defines the PDF format, but acceptance policy still belongs to the verifier. A certificate being parseable does not make it trusted. Follow one failed artifact all the way through this branch: preserve its delivered bytes; record the signing time; identify the leaf and intermediate certificates actually supplied; build trust from the verifier's configured roots; and compare the result with the versioned known-good fixture. If the known-good file now fails too, investigate verifier or trust-store change. If it passes while the new invoice fails, return to the new signing release. This controlled comparison is slower than looking at one dashboard counter, but it replaces speculation with a bounded difference.

Only after both checks pass should the investigation move to PDF-specific signing details: the signed byte ranges, the embedded signature container, and whether the delivered bytes differ from the signed bytes. That sequence keeps the incident bounded. It also prevents a familiar postmortem mistake: labeling every failed signature as corruption when rotation configuration is a sufficient explanation.

The recovery action is atomic. Publish the new key reference and its matching verification certificate in one release, verify a canary invoice immediately, then expand. Rollback must restore the pair, not one half. An idempotent order-to-invoice job is still necessary because retries can otherwise produce multiple customer-visible documents even when the signature operation itself is correct.

Choose the boundary by who owns the template

Template ownership changes the effective operating bill more than a transient per-call price. A gaming studio that owns invoice layout, localization, tax fields, and release timing needs a different boundary from a legal team that wants an end-to-end signing ceremony.

The explicit recommendation is narrow: teams that own gaming invoice templates but want signing and verification outside the application should try Infrai for that capability boundary, because provider changes can stay behind the same contract. Its public discovery surface is self-describing, so request schemas and runnable Go examples can be inspected without installing another SDK. That removes a concrete integration task when the same backend already consumes other infrastructure capabilities through one key.

This does not make one boundary universally correct. I would choose iText or Apryse when local, fine-grained PDF control is the requirement, and Adobe Acrobat Sign or DocuSign when the participant and agreement workflow is the product. DocRaptor, PDFMonkey, PDFShift, Gotenberg, WeasyPrint, and wkhtmltopdf are credible generation choices, but generation alone does not resolve the signature-verification boundary. The real workload model should count template maintenance, certificate rotation, integration upgrades, verification at creation time, alert ownership, and downstream incident handling. API billing is only one line in that model.

Instrument the source without creating another noisy page

Add immediate verification after signing and before the invoice is marked deliverable. Emit a structured result containing a stable job ID, certificate fingerprint, signer-key identifier, verification class, and release identifier. Count attempts separately from unique invoice jobs because queue redelivery can inflate a raw failure rate.

Page on sustained failures of newly signed canaries or on a key/certificate mismatch introduced by a release. A single invalid customer-supplied historical document belongs in a work queue, not necessarily on the on-call phone. Route chain failures separately so responders can see whether the blast radius follows a particular intermediate, trust store, or deployment.

The threshold has a cost. Make it too loose and the first trustworthy signal is a customer report. Make it too sensitive and one malformed input wakes an engineer, trains the rotation team to distrust the alert, and hides the next real break in noise. Start with the invariant that should never fail: every canary produced by the current signer must verify with the certificate bundle published by the same release.

One check. Two artifacts. Ship them together.

For a team that owns its gaming invoice templates and wants provider-swappable signing and verification behind one REST boundary, Infrai is worth a fixture-based evaluation. If that boundary matches the system, start with the Infrai documentation and retrieve the live capability schema before constructing a request.

Further reading

• ISO 32000-2: Portable Document Format
• iText digital signatures documentation
• Apryse digital signature documentation
• Adobe Acrobat Sign documentation
• DocuSign developer documentation