Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 7 min read

Is AI Making You a Worse Programmer? Anthropic's Own Study Says 50% vs 67%

The industry's plan for AI coding fits on a sticky note: the agent writes, the human reviews. The trouble is that reviewing is the skill you lose first when you stop writing, so the plan slowly eats the safeguard it depe

The industry's plan for AI coding fits on a sticky note: the agent writes, the human reviews. The trouble is that reviewing is the skill you lose first when you stop writing, so the plan slowly eats the safeguard it depends on.

If you've been asking yourself "is AI making me a worse programmer?", my answer is probably yes, and you'll notice it in code review long before you notice it in your own code. Anthropic's own randomized trial found developers who learned with an AI assistant scored 50% on a follow-up quiz against 67% for those who coded by hand, and the widest gap was on debugging. Christian Ekrem said it bluntly in a post this week, If you stop writing code, you also get worse at reviewing it. It's sitting at 387 points and 180 comments on Hacker News, and the thread is a good map of where people disagree.

I mostly agree with him, though a few people in that thread made points I had to think about. Further down are five habits I think actually help.

Ekrem's argument: review is a writing skill

Ekrem's post has no data. It's analogies: a jazz guitarist who stops playing and only directs other musicians, a writer who stops writing and only gives notes on AI-generated prose. His line about developers who say they review everything the agent produced: "at this point you're not really reading half of it, are you?"

I think he's right that review isn't some separate muscle. When I catch a bug in a diff, it's usually because I've written that bug myself. I know the off-by-one in pagination because I shipped it in 2019, and I know a retry loop without jitter will hammer a recovering service because I got paged for one. Most of my reviewing is pattern matching against those scars, and I got every one of them by writing code.

One HN commenter, noduerme, put it in drive-through terms: you still check the bag before driving off, but that check is worthless once you can no longer tell fries from nuggets.

Anthropic's study: 50% vs 67%, and the biggest gap is debugging

There is real evidence now, and it points the same way. In January 2026 Anthropic published a randomized trial, How AI assistance impacts the formation of coding skills (paper). 52 engineers, whom Anthropic describes as mostly junior, learned Trio, a Python async library they hadn't used, then took a quiz.

  • The AI group averaged 50% on the quiz. The hand-coding group averaged 67%.
  • The AI group finished about two minutes faster, and that difference wasn't statistically significant (p = 0.391).
  • The largest gap between the groups was on debugging questions.

Debugging is about as close to code review as a quiz gets: look at code, find what's wrong. The hand-coding group hit more errors along the way, and the authors' hypothesis is that resolving those errors on their own is what built the skill. The AI group mostly skipped that part.

It also matters how people used the assistant. Participants who handed the whole thing off scored 39%. The "iterative AI debugging" group, who pasted each error back into the chat, scored 24% and were also the slowest. People who generated code and then asked questions until they understood it scored 86%. The sample sizes per pattern are tiny (2 to 7 people), and the authors say they can't claim causation. I still find a spread from 24% to 86% hard to wave away.

The authors also note their setup was a sidebar chat assistant, and they expect agentic tools like Claude Code to have effects that "are likely to be more pronounced."

You won't notice your skills slipping

What worries me more is a result from METR. In their 2025 study of 16 experienced open-source developers, AI made them 19% slower, but afterward they still believed it had sped them up by 20%. METR now flags those results as out of date and has published newer data on late-2025 tools. The speed numbers may not hold up, but I'd hang on to the perception gap: developers are bad at judging what AI is doing to their own work.

I'd expect skill decay to work the same way, since nobody wakes up feeling rusty. You find out in a live-coding interview, like kstenerud on HN, who has 30 years of professional experience: the algorithm and the tradeoffs were no problem, but writing it by hand "felt like I was walking in molasses." Or you find out when a bad diff gets past you and you can't explain why you approved it.

The best counterarguments, and where they fall short

The HN thread pushed back hard, and some of it deserves an answer.

"Nobody reviews the assembly either." Compilers are deterministic and their output gets verified across billions of runs. Your agent's output is a fresh draft every time, written by something that is sometimes confidently wrong. We stopped reading assembly because the compiler earned that trust, and LLM output hasn't.

"Review architecture, not lines." Fair for some code. But one commenter, deltarholamda, had the free version of Grok write Ansible scripts that kept overwriting each domain's Let's Encrypt cert with the next one in a loop. That bug lives in the lines, and according to deltarholamda, spotting it took a deeper understanding of nginx, Let's Encrypt and the filesystem.

"Principal engineers don't code and still review well." Some do, but they're coasting on a decade of having written code first. Someone who starts their career prompting an agent won't have that decade.

"Reviewing lots of AI diffs is practice." Partly true, and it's why I don't think the answer is to give up AI. Reading builds some skill. Reading without ever having to produce anything builds a lot less.

5 habits that keep your review skills sharp

You can keep using AI for all five. They're about choosing where you still do the reps yourself.

1. Write the tricky part yourself first

Concurrency, money math, auth, cache invalidation: write a rough version before you prompt. Then compare it with what the agent produced. The diff between your version and its version is the most useful code review you'll do all week.

2. Review the diff before you read the explanation

Agent summaries prime you. "Added retry logic with exponential backoff" makes you see exponential backoff even when the code doesn't do it. Read the code cold, write down what you think it does, then read the summary. In practice:

# look at what changed before the chat window tells you what to think
git diff --stat
git diff -- src/   # read it, form an opinion, then open the agent's summary

DanielHB on HN does a version of this: stage files before every prompt, then check the diff afterwards. DanielHB also estimates that keeping prompts small and reviewing each step gets you about 1.5x manual-typing speed instead of the 50-100x of full vibe coding. I'd take that trade. With git it's three commands:

git add -A          # before the prompt: snapshot where you are
# ...let the agent run...
git diff            # only what the agent changed since the snapshot
git status --short  # new files show up here as ??, not in git diff

3. Have days with the assistant switched off

One day a week, or one ticket a sprint, no generated code at all. Better to find out how rusty you are on a Tuesday afternoon ticket than during an incident.

4. Track what slipped past you

Keep a plain file. Every time a bug reaches staging or prod in code you reviewed, write one line: what it was, whether a human or an agent wrote it, and why you missed it.

2026-11-18 | agent | off-by-one in cursor pagination | trusted the test it wrote
2026-11-20 | me    | missing null check on webhook payload | skimmed, Friday

After a month you'll know whether your review is getting worse and on what kind of code, which is more than most teams measure.

5. Ask until you understand, not until it works

Going by the Anthropic numbers, this is the cheapest fix. Pasting errors back into the chat produced the worst scores. Asking why the code works produced the best. If you can't explain a line to a teammate, don't approve it.

So, is AI making you a worse programmer?

If you only prompt and approve, probably yes, and your reviews will go first. "AI writes, humans review" only works if the human can still write. Otherwise the review step is a rubber stamp that happens to cost a salary.

When did you last write a non-trivial function without an AI assistant, and did you notice you'd gotten rusty? Or do you think review really is a separate skill and I'm wrong? Tell me in the comments.

Sources

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.