AI Code Feels Done Until You Refactor It

AI-generated code passes review because it looks finished. Then a teammate renames a field, changes an API response, or adds a parameter, and the code breaks in ways its tests never caught. The problem is not that the code is wrong today. It is that nobody, including the model, holds its assumptions in their head.

By the end of this article you will know the three failure modes that make AI code fragile, how to spot each one in review, and how to enforce two of them automatically in CI.

The real defect is hidden coupling

When a large language model writes a function, it optimizes for the prompt, not for your architecture. It reaches for the fastest path to working output: direct field access, hardcoded strings, a second copy of a helper that already exists three directories away.

That code works on day one. It fails on day thirty, when the schema changes and the duplicated helper in utils/billing.py does not get updated because nobody knew it existed.

Fragility is not a style issue. It is the gap between what the code does and what your team can safely change.

Three failure modes to check in review

Most fragile AI code falls into one of these buckets:

• Divergent duplicates: the model re-implements logic that exists elsewhere, and the two copies drift.
• Invented APIs: it calls methods or config keys that look plausible but do not exist in your stack, hidden behind a try/except that silently swallows the failure.
• Absent contracts: it skips type hints, validation, and error paths because the happy path was all the prompt asked for.

The first two are mechanical. You do not need a human to catch them.

Block duplicates with a CI check

A copy-paste detector finds the re-implemented helper before merge. jscpd works across most languages and installs as a single dev dependency.

# .github/workflows/duplication.yml
name: duplication
on: [pull_request]
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npx jscpd src --min-tokens 40 --reporters consoleFull --fail-on-duplication

The --min-tokens 40 threshold matters. Set it too low and you flag every data class. At 40 tokens you catch real copied logic, which is exactly what the model produces when your prompt says "add a function that formats invoices."

Catch invented APIs with type checking

The silent-except failure mode dies the moment your types are strict. A call to user.profile.settings.theme fails mypy if profile can be None, and fails outright if the attribute does not exist.

# What the model writes
def get_theme(user):
    try:
        return user.profile.settings.theme
    except Exception:
        return "default"

# What mypy forces
def get_theme(user: User) -> str:
    settings = user.profile.settings if user.profile else None
    return settings.theme if settings else "default"

The refactored version is explicit about every fallback. When the settings shape changes, the type checker tells you exactly where to look. Add mypy --strict src to CI and invented attributes become build failures, not runtime surprises.

Write prompts that name your contracts

Review gates catch the damage after it exists. You can reduce it at the source. The single highest-leverage change to your prompts is pasting the relevant types and existing helpers into the context.

Use the `format_invoice` helper in src/utils/billing.py.
Here is the Invoice type and the helper's signature: [...]
Do not add new dependencies.
Return early on missing fields; no broad except blocks.

A model with your real signatures in context has no reason to invent an API or duplicate a helper. A model without them will guess, and the guess will look confident.

What human review still owns

Automation cannot judge whether the code belongs in your architecture. A function can be duplication-free, fully typed, and still wrong: in the wrong layer, holding a connection it should release, or quietly coupling your billing module to your auth module.

Read AI code the way you read a junior engineer's pull request: ask what assumptions it makes, and what happens when each one changes. If you cannot answer that in a minute, the code is fragile regardless of who wrote it.

Key takeaways

• Fragile AI code is usually hidden coupling: duplicates, invented APIs, and missing contracts.
• jscpd with a ~40-token threshold catches re-implemented logic before merge.
• Strict type checking converts silent except fallbacks into build-time failures.
• Pasting real types and helper names into prompts removes most of the guessing at the source.
• Human review should focus on architectural fit, not formatting, since machines handle the mechanical checks.

Source

AI usage generates a fragile codebase -- this article builds on its core claim by naming the specific failure modes and adding CI checks and prompt patterns to address them.

Support this work

These write-ups are researched and published with no paywall, sponsor, or tracking. If one saved you an afternoon, a small tip keeps them coming.

USDT, USDC or USDD · TRC-20 (Tron)

TFTNsfyomKrnUutRjBTGVULp19ByW29KbY