Why Your Agent-Generated Video Frames Are Silently Corrupt (and How TypeScript Can't Save You)

At 3:14 AM on a Tuesday, our video rendering pipeline started producing corrupted timelines. Agents generated frames with IDs like Frame:Intro, frame:intro, and frame:intro (trailing space). The renderer successfully produced videos. The artifact store indexed them. The timeline service disagreed about which frames existed. By morning, 40% of videos had missing transitions, and customer support was fielding complaints about "jumpy" output.

The root cause wasn't the agent. It was an architectural assumption: that string-based frame identifiers would "just work" across package boundaries. They don't. Not when multiple services interpret the same contract independently, and not when agent output is treated as trusted input.

I'm working with heygen-com/hyperframes, which provides a clean boundary: describe HTML, render it into video, expose the workflow to agents. That boundary is powerful, but it creates a specific failure mode. The same frame definition gets interpreted by composition logic, rendering engines, artifact stores, and orchestration layers. If those interpretations diverge, the system produces output that looks correct but isn't.

Type Safety That Actually Prevents Production Failures

The first fix was rejecting invalid frame IDs at compile time. No runtime check catches an agent that occasionally adds whitespace. TypeScript's template literal types can enforce a stable format:

type ValidFrameId<S extends string> =
  S extends `frame:${infer Body}`
    ? Body extends `${string}-${string}`
      ? Body extends `${string} ${string}`
        ? never
        : S extends `frame:${Lowercase<Body>}`
          ? S
          : never
      : never
    : never;

export function defineFrame<const Id extends string>(
  id: Id & ValidFrameId<Id>,
) {
  return id;
}

type Frame =
  | {
      kind: "html";
      id: ReturnType<typeof defineFrame>;
      html: string;
      durationMs: number;
    }
  | {
      kind: "transition";
      id: ReturnType<typeof defineFrame>;
      from: ReturnType<typeof defineFrame>;
      to: ReturnType<typeof defineFrame>;
      durationMs: number;
    };

export function renderFrame(frame: Frame): string {
  switch (frame.kind) {
    case "html":
      return frame.html;
    case "transition":
      return `${frame.from}->${frame.to}`;
  }
}

const intro = defineFrame("frame:intro-opening");
// const invalid = defineFrame("frame:Intro Opening"); // compile-time error

This utility enforces frame:<lowercase-slug>-<lowercase-slug> at compile time and preserves discriminated unions for exhaustive narrowing. It deliberately avoids deeply recursive conditional types. I once inherited a registry type that recursively walked nested frame trees and inferred every possible event path. It worked in examples. In production, adding several hundred frames caused TypeScript compiler memory exhaustion and made editor diagnostics unusable. The fix was bounding type-level computation and moving recursive validation to runtime.

Type safety protects maintainers. Runtime validation protects the system. Agent output is untrusted input, even when the agent uses a strongly typed SDK. Validate IDs, HTML policy, durations, and resource references before creating a frame contract.

Package Boundaries That Reflect Execution Boundaries

The second fix was making dependencies unidirectional:

packages/
  frame-contracts/
  composition/
  renderer/
  artifact-store/
apps/
  render-worker/
  agent-gateway/

frame-contracts must not import composition. If a type appears to require that import, the contract is carrying behavior that belongs elsewhere. This prevents circular internal imports. Another production failure involved composition importing a helper from renderer, while renderer imported a normalization function from composition. Incremental builds became nondeterministic: a clean build passed, but watch mode occasionally consumed stale declarations and produced missing exports. The solution was extracting pure normalization into frame-contracts or a small leaf utility package. Do not solve circularity with barrel files alone; barrels hide the cycle rather than remove it.

Cache Keys That Include the Inputs That Actually Matter

Rendering tasks depend on browser versions, fonts, environment variables, and generated assets. A cache key that includes only source files is unsafe. This turbo.json makes task inputs explicit:

{
  "$schema": "https://turbo.build/schema.json",
  "globalDependencies": [
    "pnpm-lock.yaml",
    "tsconfig.base.json",
    "biome.json",
    "toolchain/renderer-version.txt"
  ],
  "tasks": {
    "build": {
      "dependsOn": ["^build"],
      "inputs": [
        "$TURBO_DEFAULT$",
        "!**/*.md",
        "!**/*.test.ts",
        "!**/__fixtures__/**"
      ],
      "outputs": [
        "dist/**",
        "build/**",
        ".next/**"
      ],
      "env": [
        "NODE_ENV",
        "RENDERER_VERSION",
        "FONT_BUNDLE_VERSION"
      ],
      "cache": true
    },
    "typecheck": {
      "dependsOn": ["^typecheck"],
      "inputs": [
        "$TURBO_DEFAULT$",
        "tsconfig.json",
        "../tsconfig.base.json"
      ],
      "outputs": [],
      "cache": true
    },
    "lint": {
      "inputs": [
        "$TURBO_DEFAULT$",
        "biome.json"
      ],
      "outputs": [],
      "cache": true
    },
    "test": {
      "dependsOn": ["^build"],
      "inputs": [
        "$TURBO_DEFAULT$",
        "vitest.config.ts"
      ],
      "outputs": [
        "coverage/**"
      ],
      "env": [
        "NODE_ENV",
        "TZ"
      ],
      "cache": true
    }
  }
}

Turborepo hashes declared inputs, dependency outputs, and relevant environment values before consulting the remote cache. FONT_BUNDLE_VERSION isn't cosmetic: changing fonts changes rendered pixels, so it must participate in the hash. The renderer version belongs in the same category. More inputs reduce false cache hits but increase cache misses. For video rendering, correctness wins. For pure type checking, keep inputs narrow because the output is declarations and diagnostics rather than pixels.

The Trade-Off No One Talks About

This architecture is less convenient than a single package that accepts a prompt and emits a video. That's intentional. Hyperframes remains the rendering-oriented core. The surrounding monorepo makes contracts, orchestration, and execution independently testable. The payoff is predictable incremental builds, narrower failure domains, and agent integrations that cannot silently invent invalid frame identities.

But here's the question I haven't solved cleanly: where do you validate frame content policy? Agents can generate syntactically valid IDs and semantically nonsense HTML. Do you reject it at the contract boundary (composition fails fast, but agents get cryptic errors)? Do you accept it and flag it during rendering (agents get better feedback, but you've already committed to storage and queuing)? Do you run a separate validation pass before composition (adds latency, duplicates logic)?

I've tried all three. Each one trades off a different production pain point. What's your architecture, and which failure mode did you choose to live with?