Claude Said "Done as Instructed" but Wasn't Following Me. It Was Following Old Specs It Had Saved in Its Own Memory and Rules

This is an English version of an article first published in Japanese on Zenn.

Introduction

If you use Claude Code for a long time, you may start to feel that "it doesn't understand me like it used to" or "it suddenly got worse at picking up what I mean." Claude says "I did it as instructed," but what comes out does not follow the instructions.

That happened to me too. I first noticed it in September 2026. Claude kept trying to use a process I had already removed from my product. At the time, I asked Claude something like this (wording lightly adjusted):

This feature is already retired, so why is it still around? Is something old left in memory?

The cause was an old note that had quietly stayed in auto memory. I cleaned up the memory and thought the problem was solved.

But in October, while Claude was fixing the analysis screen of a dev tool I build myself, the same thing happened again. Every time it fixed something, the screen went back to an old form I thought had already been fixed. When one part was fixed, another part got rebuilt, and only my tokens went down. Around then, I told it this:

I've asked many times, but I want it at the same granularity as the graph. How do I say this so it gets through?

I asked it to change only the text, and the whole screen got rebuilt. When I asked about a different part, the text changed instead. I no longer understood what was going on.

This time it was not memory. An old spec had quietly been written into an instruction file that is read every time: a rule (files in .claude/rules/).

In my setup, the Claude I talk to (below, "the main agent") splits up the work, and the actual edits are done by other AIs it hands the work to (below, "subagents").

Afterwards, I went back through the work logs from that time, built a similar situation on a small screen, and ran it 155 times on the current Claude.

Even when an old spec stayed in memory or a rule, the current Claude followed the correction whenever the new correction came into view. Out of 155 runs, the number of runs that read the correction and still went back to the old form was 0. In 19 of 20 runs, Claude itself rewrote the memory holding the old spec to match the current spec. The only runs that went back were the ones where the correction was out of reach.

In this article I describe what actually happened, the results of the reproduction experiment, and the structure I changed afterwards.

What actually happened

1. A retired path kept coming back from old memory (September)

When I asked an external AI for a review, I used to put a relay mechanism in between. On September 12 I retired this relay path from the product, but the main agent kept using it after that.

The cause was a note in auto memory dated September 4: "Use this path by default for external reviews." There was a document recording the retirement, but it covered the product's feature. It did not touch the path the main agent used when asking for reviews. The main agent treated the memory as the truth.

Looking back now, I remember asking for it to be removed again and again, and it still stayed (the number of times it came back is not in the records).

2. A one-off exception was written to memory as a permanent policy (late September)

In late September, a check that stops contradictions between documents was too strict and was blocking even valid writes. What I meant was: redesign it, drop it if it isn't needed, keep it if it is. With that in mind, I removed the check for the time being.

But the main agent wrote this down as a permanent policy: "Don't add mechanisms that find contradictions." It wrote it in both the decision log (a file where decisions are collected) and auto memory. Later conversations treated this policy as the truth.

On the same day, September 29, I cleaned up memory. I cut the notes from 26 files and 590 lines to 19 files and 269 lines. I thought that settled it.

3. I thought I had cleaned up memory, but it was in a rule too (October)

When I cleaned up memory, I also remember asking whether anything similar was left elsewhere. Even so, next it came out of a rule. When you are looking for "old notes in memory," copies that went into some other place don't come into view.

This is the October analysis-screen case. The correction I had given many times, "list the rows of the analysis screen in the same units as the graph screen," was written in the decision log.

Meanwhile, the instruction file loaded every time (a rule in .claude/rules/) still held an old copy of the screen spec. According to the records, the subagent fixing the screen read this rule as the correct spec every time and put the screen back into the old form. Whether it followed the decision log or the rule, the result would conflict with one of them.

It was the main agent's own instructions that had put that copy into the rule. Back then, the main agent looked back and said "I had been giving the same wrong instruction myself." It admitted that the instruction it had just given the subagent was to rewrite the rule to the new spec.

CLAUDE.md had this rule: "When the user decides something, rewrite the rule and spec files it binds in the same change." The aim was: "When I decide something, also fix the rule and spec files related to that decision at the same time." It existed to prevent a state where the decision lives only in the log and the rule read every time stays old. But according to the main agent's explanation, it read this as "write the spec into the rule" and told the subagent to do so. A rule meant to stop copies was misread and created more copies.

The main agent gave detailed instructions. Even so, it wrote, the subagent treated the old text left in the repository as the truth over the instructions it was given.

4. There was no "correct side" to go back to

The decision log, the rules, and auto memory were all written by the AI. And they contradicted each other. Clean up memory, and the same thing enters a rule. It was past the stage where stopping one of them would bring back a correct state. That is because there was no correct side to go back to anywhere.

In the early hours of October 8, I said that memory was causing quite a few bugs, and I turned off auto memory.

How this differs from a similar existing article

kamo78's article "If Claude Code's performance seems to drop, suspect auto memory" (in Japanese) covers the link between Claude Code's performance drop and auto memory in detail.

It is a case where three notes Claude saved on its own contradicted the rules of workflows and skills and a policy that had already been removed, leading to symptoms such as reviews not converging and designs becoming excessive. The author turned off auto memory, kept the rules in CLAUDE.md and skills, ran the same work about 4 times, and reports that it did not happen again (also carefully noting that this is not a statistical proof).

I think the root is the same. The difference is "whether a correct side remained." In kamo78's case, the rules remained as the correct side, so turning off memory let things go back there. In my case, there was no such side.

So when an old instruction and a new correction sit side by side, how does the current Claude actually behave? From here on is the experiment that checked this.

Reproduction experiment

I had one guess. It was my reading in hindsight.

I meant "add what's missing from the current spec." But maybe the AI doing the work took it as "additions to the old spec," and, with good intentions, filled the gap between the old spec and the current screen on its own.

To check this, I built a small screen and ran it 155 times under different conditions.

Experiment conditions: On a small screen made of 4 files, I ran the Claude Code CLI with one default model (Opus 5.5) and one configuration. Each condition ran 5 to 10 times. For auto memory, I did not use the real mechanism. Instead, I wrote the note text directly into the instructions to Claude. For delegation to a subagent, I did not use the real thing either. Instead, I wrapped the request in a delegation-style instruction and ran it once. Scoring compared the diffs of the changes by machine, and when the match failed, I checked the diff by eye. The total cost was about $19.

What I prepared

• Current screen: a table with one row per graph point (node), filtering, sorting by number of connections, and linking between the table and the graph.
• Old spec: one row per bucket of connection count (0-1, 2-3, 4 or more), a total row, CSV export, and pagination.
• Rule: one version describing the current spec, and one describing the old spec.
• Decision log: a log containing the correction "Dropped the old form; now one row per point." Besides a short 14-line version, I also made a long version. The real log was about 500,000 bytes including older entries, so I stretched it to about 260,000 bytes and 5,948 lines and buried the correction around line 3,132. I also tried cases where CLAUDE.md has a line pointing to this log, where it doesn't, and where the log itself is absent.
• Old memory: a note that described the old spec as "the user's correction."
• Delegation instructions: imitating what the main agent passes to a subagent, a short preface before the request saying that this is a fix to the analysis screen, that the spec is as written in the rule file (.claude/rules/ui-spec.md), and that the only file it may change is the screen file (index.html).

There were 3 kinds of requests.

• Request 1 (specific): Add a column with each row's last-updated time to the current screen.
• Request 2 (vague): Add what's missing from the current spec.
• Request 3 (specific + vague): I also want to see each row's last-updated time. Add what's missing from the current spec.

Results

First, the overall picture.

All 18 runs that went back to the old form were runs that did not read the correction.

The next table breaks down the 85 runs where the rule held the old spec. The other 70 runs were controls where the rule held the current spec; I describe them below the table.

In the conditions where the rule held the current spec (controls, 70 runs in total), no run went back to the old form. Machine scoring counted one run as going back, but looking at the diff, it was a matching error. However, in 3 of the 10 runs with no log and Request 2, Claude added small improvements I hadn't asked for.

1. When the correction came into view, Claude did not follow the old instruction

If the decision log was in the repository, Claude searched for it and read it on its own, even without a pointer in CLAUDE.md.

The long log was the same. Looking at 20 runs, 10 with the old rule and 10 controls, the first read cut off around line 1,271. But in 19 runs, Claude then searched for headings and words and re-read the relevant section by specifying its position. The remaining 1 run read around the correction that showed up in the search results and stopped there.

In the conditions with the old spec in memory, all 20 runs (10 with the old rule and 10 controls) read both the note and the log and judged the note to be older. In 19 of 20 runs, Claude rewrote the note to match the current spec.

2. With no correction and only the old rule, vague requests went back to the old form

With no log and only the old rule, vague requests went back to the old form in 13 of 15 runs. Claude added rows per bucket, totals, CSV, and pagination, and in many runs removed the current screen's "one row per point" table and sort order.

In one run, the work report said "I also added features that were in the spec but not yet implemented." Treat the old spec as the truth, and fill the gap with the current screen on its own. The behavior I had guessed appeared exactly as is in this condition.

The tricky part is that Claude itself believed it was following the current instructions. It did not report breaking the screen. It reported finishing it to spec. I think the reason I felt back then that "it suddenly stopped understanding me" was not that what I said was ignored. It was that a different spec was being followed correctly.

3. With a specific request, even the old rule alone did not cause a revert

In the same "no log, old rule only" setup, when I asked specifically, "Add a column with each row's last-updated time," Claude added only the column in all 5 runs.

4. Adding "the spec is as written in this file" sometimes stopped Claude from looking

When the log was in the repository but I added the delegation instructions and CLAUDE.md did not point to the log, Claude went back to the old form in 5 of 10 runs. None of those 5 runs ever opened the log. They read just the one screen file and rewrote it right away (the rule is loaded automatically). The other 5 runs, which opened the log, did not go back.

With the same parts, when I made the request without the delegation instructions, Claude searched for and read the log on its own in all 10 runs. If CLAUDE.md had one line pointing to the log, Claude read the log in all 10 runs even with the delegation instructions.

What I learned (my reading)

From here on is my reading based on the experiment.

When the current Claude sees both an old instruction and a new correction, it picks the new one. Old text does not win by default.

I think old instructions take effect when the new correction is out of reach. In this experiment, that happened in three forms.

1. The correction exists only in a place nothing points to. The rule read every time holds an old copy, and the correction exists only in another file.
2. The requester points to one place: "the spec is in this file." The one receiving the request sometimes skips looking elsewhere (this is a comparison of 10 runs each, so I can't say it for certain).
3. The request is vague. When you leave the standard up to it, as in "add what's missing," it treats the old spec in view as the truth and starts filling the gap.

That said, the conditions differ from the real analysis screen. Back then, the main agent's instructions named the decision log and said "read it," and CLAUDE.md also pointed to the log. As far as the records show, two things overlapped: copies kept entering the rule, and the main agent's own instructions pushed toward more copies. So this experiment does not directly explain what happened back then. Please read it as a check of where things break and where they don't when the same shape is built on the current Claude.

The stale-copy problem happened outside the screen too

The only case recorded as a "regression" is the analysis screen. But the same pattern, where something copied to another place goes stale and causes errors, happened many times outside the screen too. Here are the ones with numbers on record.

• When I collected and classified 246 stale or wrong statements, the most common kind was "copies." A subagent collected them from the git history and decision log of my dev tool and its predecessor. Copies of code, values, strings, and lists were 88, and copies of states, versions, and counts were 42, together more than half. The rest: 43 were wrong when written, 35 were just-in-case statements, 34 pointed to things that no longer existed, and 4 were unknown (the classification was done by the main agent, with no third-party check).
• CSS dropped out of a copied list of distribution files, and I shipped 2 versions broken. I was copying the list of files to ship into another place by hand, and a new CSS file was missing from the list.
• The same thing happened not just in documents but also in code calculations. The same value was computed differently on each screen, and out of 842 subagents, only 4 had values that matched across screens. The analysis screen shows, for each subagent, "how long it was running (elapsed time)." Each screen computed this value with its own calculation. Reviewers read it 3 times and didn't find it. It came out when I ran the 176 conversation logs from that point (with 842 subagents appearing in them) through each screen's calculation and compared them. Each fix, when re-compared, revealed a new mismatch, and the number of subagents with mismatched values went down from 27 to 2 to 0. Similar to how things came out of rules after I cleaned up memory, lining up and comparing actual values and behavior found more than reading and searching for the same kind of thing.
• I deleted 4,227 source code comments in one go. Comments also become stale copies when the code changes. Even after deleting them all, the built code matched the code before deletion, and the TypeScript type check passed. Only 11 notes about user data and external constraints were moved into the shape of the code or into spec files (I did not count whether contradictions went down after the deletion).

From written rules to enforcement

In fact, the rules "keep a fact in one place and don't copy it" and "put a correction right away into the file it binds (because a correction written only in the decision log loses to the next task that reads the original rule, and things go back)" were already in CLAUDE.md before the analysis-screen case. It still happened.

The same goes for memory. Claude Code's official docs (How Claude remembers your project) separate the roles of each location. CLAUDE.md and .claude/rules/ are where people write "instructions and rules" (build steps, conventions, rules like "always do X"). Auto memory is where Claude writes "what it learned" on its own (user preferences, corrections it received, context you can't tell from the code).

In my setup, auto memory contained the rules themselves, like "Use this path by default for external reviews." (The case in section 2, "a one-off exception became a permanent policy," is separate: that was a misreading of what I said.)

From here on is a guess. Claude probably wrote the corrections it received from me into memory as "what it learned," just as the official docs describe. Remembering corrections is a good thing in itself. But spec and rule content slipped into those corrections. Specs and rules go stale as soon as they change. For things in memory, it is hard to see when and based on what they were written, so they stayed unfixed when the spec changed and kept being read as correct.

After the damage, I asked several times for memory and rules to be cleaned up. I once said "reject anything at all that can go stale," turned off memory, and moved its contents to the right places. When I turned memory back on, I also wrote "I've taken stock many times" (both lightly adjusted in wording).

So, would it have been prevented if things had been placed according to the official role split? If Claude remembered it as a correction, then by the official docs, memory is the right place. From Claude's point of view, it was already placing things as the docs say. The official yardstick is "a rule a person wrote, or something Claude learned." What I wanted was "whether it goes stale easily." Only things that are "a correction received, but spec in content" get sent to different places by the two yardsticks. I think it is precisely because this is a mechanism that remembers with good intentions that it also picked up things that go stale easily.

I strengthened the countermeasures bit by bit, after each round of damage. At first there was nothing to stop it. First I added written rules, then I added a hook that watches the locations. But the watched scope did not include copies of specs, and the analysis-screen case slipped through there. After that, I made it so only the main agent can write memory, stopped memory writes of the "correction" type, and widened the scope several times.

Honestly, checking it myself every time was exhausting, so I left it to hooks. Asking for cleanups or writing rules down in text does not stop it. That is the same conclusion as the analysis-screen case.

Note that this is not limited to personal development. Business rules change. Requirements get revised. In development, specs change. This is common at work. If corrections and decisions received before the change stay in memory or rules, you may end up in exactly the same situation. Claude will faithfully keep the old spec as correct and report "I did it as instructed."

So what I added after that was not written rules but mechanisms that stop things without relying on judgment.

1. Automatically attach the authoritative spec on every delegation. When the main agent hands work to a subagent, a hook (a Claude Code mechanism that automatically inserts a predefined process each time a specific action happens) attaches the current spec text. The current spec comes into view before the subagent reads an old copy.
2. Fail spec copies by machine. If a document contains a copy of the screen spec, a check fails.
3. Narrow memory writes with a hook. A hook stops writes from subagents, notes of facts without a way to verify them, and so on. I turned memory off once, and turned it back on with this hook in place.
4. Don't write comments by default. A comment that restates the code becomes a second copy.

I have not counted how much rework went down after adding these 4. Please read this not as a list of countermeasures with proven effects, but as a record of closing off, in the structure, the causes I saw in the records and the experiment (copies remaining, and corrections not reaching).

There is one more thing the experiment seems to suggest. It is better to pass subagents not only the spec text but also a path to the decision log. I have not built this into the structure yet.

Caveats

• Few runs and models. Each condition ran 5 to 10 times, with one model and one configuration.
• Small parts. There were 4 files, and even the long log had headings per section, a shape easy to hit by search. I did not test repositories with many files or logs whose headings are not split by topic.
• Memory and delegation were stand-ins. Auto memory was inserted text, and delegation was stood in for by a single run.
• I could not recreate the conditions back then. Back then, memory writes had no stopper, and old notes and copies were in many places. As I remember it, around August there was a case where "it was also reading the correct spec, but memory took priority." But almost no conversation logs from that time remain, and I could not confirm it.
• That much of the "it got worse" feeling comes from this is my reading. I do think there is a pattern where text piles up in instruction files and memory the more you use them, so the longer someone uses it, the more likely they hit contradictions.

Conclusion

When the same correction doesn't stick no matter how many times you give it, it is easy to think the model got worse, or that you need to write more forcefully.

But the more faithfully an AI follows the instructions it reads, the more it looks to a person as if "it suddenly stopped understanding me" when those instructions are old.

In these 155 runs, no run where the new correction came into view ever went back to the old form. They went back only when an old copy was in a place read every time and the new correction did not reach the one doing the work.

After I cleaned up memory and added mechanisms to stop copies, the stress of repeating the same thing over and over almost disappeared. This is not a counted result. It is how it feels to me.

If you feel "it suddenly stopped understanding me" or "a spec I fixed keeps coming back," before suspecting the model, check two things.

• Whether old copies of specs remain in CLAUDE.md, .claude/rules/, and auto memory, which are read every time.
• Whether the correction you gave is in a place the one doing the work will see (the file that defines that spec, or a pointer that always leads there).

What needs reviewing may be less the model's intelligence and more where the instructions live, and the path to them.