Length Is Cheap, Competition Is Not: Attention Predicts Context Rot in Retrieval [R]
Author here. A short summary, since the title is a claim. Setup: key-value retrieval, 512 to 32K tokens, five open models (0.5B to 3B, four families). At equal length and equal evidence position, we fill the context eith
π
This source provides headlines only. Use the button below to read the complete article on the original site.
π° Read the original article on r/MachineLearning
Originally published by r/MachineLearning. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.