Posted on Nov 19 • AI-assisted
ForecastGym: A Live, Controlled, and Replayable Benchmark for Forecasting Agents
The prediction quality of forecasting agents relies heavily on both reasoning models and retrieved external evidence, making it difficult to evaluate each independently without future data leakage. Researchers launched ForecastGym, a live, controlled, and replayable evaluation environment. This framework combines prospective data collection with an offline replay mechanism, enabling researchers to precisely isolate the impact of evidence retrieval quality from a model's core reasoning capabilities.