RealtimeBench Tests AI Agents Under a Live Clock as GPT-6.1-sol Falls Behind Jev

A new evaluation benchmark of 180 tasks across six game types that measures LLM agent performance while the environment keeps changing during inference, rather than pausing to wait for the agent.

GPT-6.1-sol beats the faster Jev when games are paused, but Jev pulls ahead when games keep running, showing that raw reasoning strength alone is not enough in live environments.

Two-Timescale Offline Recursive Self-Improvement: a slow teacher analyzes failures and tests corrections in paused games between attempts, then writes reusable instructions for a fast student that acts without teacher calls during gameplay.

Yes, especially for applications driven by continuously changing data such as customer support, financial monitoring, and digital government services, where latency decides whether an action is correct or too late.