Your coding agent has opinions. Log the calls it made for you

Code review is good at one thing: finding mistakes. A failing test, an off-by-one, a missing null check. Something is wrong, and a reviewer can point at it.

A lot of engineering work has no mistake to find. Cache or recompute? Retry in the client or in the queue? Measure latency at p95 or p99? Both answers are defensible. Someone has to pick, and the pick shapes everything after it.

Coding agents now sit in the middle of those choices. They draft the design note, propose the approach and argue for it. A trial published by Anthropic on 27 October 2026 suggests we should check who is actually making those calls.

What the trial found

Researchers Amaç Herdağdelen, Hailey Joren and Alex Tamkin gave 50 professional analysts a realistic statistics task with Claude Code. The task had four open decisions, each with two reasonable options, such as which outcome measure to use and how to weight the data.

A coin flip set which option each analyst's agent leaned towards. The agent was told to make the strongest honest case for its lean, to change its mind if the evidence turned decisively against it, and never to misrepresent results. The analysts didn't know.

• Analysts matched the agent's lean on 86% of the four decisions. About 50% would mean no effect.
• The four coin flips alone predicted 82% of final recommendations, against a 61% baseline.
• In 38 cases an analyst stated a preference first. When the agent disagreed, they switched 60% of the time. When it agreed, 6%.
• Afterwards, 77% explained their choices with the agent's arguments. 15% mentioned the agent.
• Experience didn't predict who deferred.

The agent also helped. Of the 11 analysts who chose to work without it, six made serious methodological errors, against three of the 50 who used it. The groups weren't comparable, so treat that as a hint, not proof.

The researchers' own summary of the review problem: "Checking AI-assisted work for errors, or against a reference answer, assumes there is a correct answer."

Mapping it onto engineering work

This next part is my inference. The study used analysts on a data task, not engineers on a codebase.

Agents are now involved in exactly the kind of choices this study tested. Think of an architecture decision record drafted with an agent, or a pull request where the agent picked the retry strategy. A spike write-up where it chose the benchmark. In each case the reviewer checks that the code works. Nobody checks whether the choice was the author's.

Two habits make it worse:

1. One-sided options. The researchers found the agents rarely made a strong case for the other option. An ADR with one well-argued option and a strawman isn't really a decision record.
2. Lost provenance. If 77% repeat the agent's argument and 15% credit it, then "Considered alternatives" sections are partly the agent's memory, not yours. Nobody is lying. Memory is just generous to its owner.

The artefact: a judgement-call log

Paste this into your PR template or ADR template. Fill one row per open choice that could change behaviour, cost or a threshold. Skip trivia. The researchers warn that surfacing every decision "could quickly become overwhelming".

The example rows are illustrative. They are not from the study.

## Judgement calls in this change

| Call | Option A | Option B | Chosen | Result under the other option | Argument came from | Independent check |
|---|---|---|---|---|---|---|
| Example: retry location | Client-side retry with backoff | Queue redelivery | B | Client retry also passes tests; duplicate-write risk higher | Agent proposed B; I added the idempotency check | Reviewer read both options before seeing agent output |
| Example: latency SLO metric | p95 | p99 | p99 | p95 alert would not fire on last month's incident | Mine | Same model used by reviewer: treat as one view |
| Example: cache strategy | Cache per request | Cache per tenant | A | Per-tenant cuts DB reads in a load test | Both | Not checked yet |

How to use the columns:

• Option A / Option B: write both as if you meant them. If you can't argue the one you rejected, ask the agent for "the strongest case for option A, then the strongest case for option B".
• Result under the other option: the important column. If the outcome flips (a test, an SLO, a cost line), the reviewer should know this row is the real decision.
• Argument came from: mine, the agent's, or both. It takes ten seconds and it's the only provenance you'll get.
• Independent check: if the reviewer used the same model with similar prompts, say so. Two signatures from one model are one opinion.

A quick self-test

Before you merge, try this on one row. Open a fresh agent session, frame the question neutrally and ask for the case for the option you rejected. If you find it more convincing than you expected, keep the row and flag it for the reviewer.

This is a rough check, not an experiment. The study could measure deferral only because the leans were randomised. But it's cheap, and it shows you where your view is thin.

Limits

• The leans were deliberately strengthened. With no lean assigned, Claude Opus 4.8 made the same five choices in 112 of 123 runs, so models do lean, but the effect may be smaller in normal use.
• One task, one field, one model, about an hour long.
• Anthropic studied its own model and published the results itself.

Sources

• How AI agents shape experts' judgment calls, Herdağdelen, Joren and Tamkin, Anthropic, 27 October 2026.
• Who's Aligned to Whom? How AI Agents Shape Experts' Research Judgment Calls, full paper (PDF).

In your last merged PR that an agent helped with, which design choice would you struggle to say was yours?