Once the token limit was raised, AI feedback changed English learners' words more often

Made with AI agents working in my name. Code Apache-2.0, text CC BY 4.0. Only counts are published: no essay, code, hint or feedback text.

When AI feedback quotes a student, the quote should be what the student wrote. If the model quietly fixes a word while "quoting", the student is shown a sentence they never wrote, and their real mistake goes unseen. I have been auditing this with preregistered tests. Two new results are in.

Audit 7: English learners' words were changed more often

Audit 6 asked whether a small open model (Qwen2.5-1.5B, on CPU) quotes English language learners less faithfully than their classmates. It got no answer, because a third of its replies ran out of tokens and could not be read.

Audit 7 re-ran the test on 292 new matched pairs of PERSUADE 2.0 essays (an English learner's essay and a non-learner's essay with the same prompt and the same score). The only change was a longer token limit, 800 instead of 450. All three predictions were written down before any feedback was generated.

• English learners' words were changed more often. 15.8% of located excerpts from learners' essays changed the student's words, against 10.0% for the matched essays: +5.85 points, 95% interval +1.3 to +10.4. Verdict: held. It also holds on the 177 pairs where every reply could be read.
• Saying "this student is an English language learner" did not make it worse. It lowered the rate by 3.2 points (interval -6.2 to -0.3). The plan asked whether the change stayed within 4 points, and the interval crosses that edge, so the verdict is inconclusive.
• Long replies changed words about twice as often. Excerpts in replies over 450 tokens, the ones audit 6's limit cut off, changed the student's words 20.7% of the time, against 11.0% in shorter replies. Verdict: held.

That last result explains the earlier miss. Audit 6's underpowered estimate pointed the other way (-3.9 points), and the replies its token limit removed were the least faithful ones. A token limit does not just lose data. It can hide the worst of it.

Audit 8: programming hints in a real course

ProgFeed is an open dataset (CC BY 4.0) from a real introductory Python course. Failing submissions got, at random, a gpt-4o-mini hint, an AI-written failing test, or nothing. The hint prompt included the student's whole file and said "Do not suggest code changes." I checked every snippet in backticks in 343 hints to 119 students.

• Names are right. Only 2 of 519 variable or function names were found nowhere (0.4%).
• Code snippets are not. Of 177 code snippets, 50% are in the student's file exactly as written, 18% only with different spacing, 11% come from the assignment, and 21% are in neither the student's file nor the assignment. 29% of hints that show code contain one.

Of the five predictions, two held, two were inconclusive (67.8% of snippets found against my 70% bar, and 29.3% of code-showing hints against my 30% bar; both intervals straddle the line), and one had too few snippets for a verdict. A follow-up, 8b, is planned. It will check whether those snippets are near-copies of the student's own lines or new code, and whether they turn up in the student's final submission more often than in classmates'.

Honest limits

• Audit 7 is one small CPU model and one prompt, not the tools schools use. It shows the effect can happen, not how common it is. ELL and non-ELL essays are matched on prompt and score only.
• Three of audit 7's four runs hit a stalled machine and stopped at their preregistered time budget, so 292 pairs ran instead of the expected 450 or more. Every comparison still met its floor.
• Audit 8 is one model, one course and five labs, and only 177 code snippets. The check is literal: an unmatched snippet may be a paraphrase in code form, a corrected line or a misquote.
• No one read the quotes. The checker cannot tell a deliberate correction from a slip.

What I take from it

If an AI tool quotes your students, check English learners' quoted sentences first, and when a hint says "your line ...", find that line before changing anything. The prediction ledger now has 42 preregistered predictions: 16 held, 9 reversed and the rest mixed. It lists every one, including the misses.

Links

• Audit 7 results: https://github.com/munimtechnologies/feedback-quote-audit/blob/main/RESULTS-7.md
• Audit 8 results: https://github.com/munimtechnologies/feedback-quote-audit/blob/main/RESULTS-8.md
• Prediction ledger: https://github.com/munimtechnologies/feedback-quote-audit/blob/main/PREDICTIONS.md
• Guide for teachers: https://github.com/munimtechnologies/feedback-quote-audit/blob/main/TEACHERS.md

— MV2, Munim, Inc.