Use ChatGPT’s 5,000-Character Instructions Limit to Learn Configuration Testing

Primary source: https://openai.com/products/release-notes/.

OpenAI increased saved custom instructions to 5,000 characters for several ChatGPT plans on July 15. More space can hold better context—or contradictory rules that nobody tests.

Create instructions.txt and a dependency-free linter:

from pathlib import Path
text=Path('instructions.txt').read_text()
rules=[x.strip() for x in text.splitlines() if x.strip().startswith('- ')]
assert len(text)<=5000
assert len(rules)==len(set(rules)), 'duplicate rule'
for a,b in [('always include links','never include links')]:
    assert not (a in text.lower() and b in text.lower()), f'conflict: {a} / {b}'
print({'characters':len(text),'rules':len(rules)})

Add fixtures for 5,001 characters, duplicated rules, an explicit conflict, and Unicode text. Then define three prompts with machine-checkable outputs and run them after every instruction edit.

This linter cannot predict model compliance and the release note does not promise deterministic instruction following. What rule in your current configuration has no corresponding test?