CAPTCHAs have gone spatial — and AIs still do not see like humans. A new benchmark, TuringSpace, tests AI models with 125 interactive CAPTCHAs that demand real spatial understanding: rotating, folding, dragging and predicting what happens on screen, with clicks and drags in a real browser. The result: the best model, GPT-6 Astra, solves 74.1% of the tasks — against 94.1% for humans. The study was published on arXiv on Thursday (19/11) by researchers from the University of Pennsylvania, the University of Minnesota, the University of Southern California, MBZUAI, the University of Wisconsin–Madison and Meta.
Five spatial abilities, zero multiple choice
TuringSpace covers five abilities: spatial relations (what is behind what, what a person in the scene can see), mental transformation (rotating, folding and cutting), closed-loop manipulation (every action returns visible feedback), multi-step planning (one wrong move leads to a dead end) and spatiotemporal reasoning (tracking moving objects across up to 16 frames). The difference from other benchmarks is that there are no options to guess from: the agent must act — click and drag — and correct course with every screenshot. Each task is a program that generates fresh instances from random seeds, with an automatic checker and a reference solution, so the same set also serves as a training environment, TuringSpace-Gym.
The scoreboard: humans 94.1%; best AI, 74.1%
Three authors played all 125 tasks, averaging 33.1 seconds per task, and scored 94.1%. GPT-6 Astra, the best model evaluated, reached 74.1% — and when it must solve all three instances of each task, it drops to 58.4%, against 84.8% for humans. Claude Opus 5.5 scored 65.3% and Gemini 4 Argon 57.9%. Among open-weight models, the best was Kimi K3 at 32.0% — and no specialized computer-use agent scored above 3.5%. The hardest category for almost everyone was spatiotemporal: GPT-6 Astra solved only 52.0% of those tasks.
The typical error: the AI misreads the screen — and invents evidence to defend itself
According to the authors, most failures come from misreading the screenshot, even when it shows the result of the model's own action. Worse: reasoning rarely corrects a wrong belief formed early — in 74% of failed attempts, the model fabricates an observation that is not on the screen; in 66%, it decides based on a hallucinated prior experience; in 55%, it submits an action blindly, as a probe. And thinking more does not always help: raising reasoning effort lifted Claude Sonnet 5.5 from 29.6% to 74.4%, but Gemini 3.8 Flash stayed between 29.6% and 33.6% at every level.
Can it be taught? A 9B model trained in the gym overtook Kimi K3
To test whether the ability can be learned, the researchers fine-tuned Qwen3.5-9B on 55,000 verified trajectories generated by TuringSpace-Gym. Accuracy jumped from 1.6% to 33.1% — above Kimi K3 and Qwen3.8-Max — and the resulting model, TuringSpace-9B, improved on all seven external benchmarks evaluated, including 9.9 points on real-world indoor videos (VSI-Bench). In other words: training with on-screen actions transfers to real-world scenes. The authors promise to release TuringSpace for other researchers. The study is an arXiv preprint — it has not yet been peer-reviewed — and the authors themselves note that the tasks are synthetic. This week, Geekiki also reported on OpenAI shielding GPT-6.2 Astra against adversarial distillation: OpenAI shields GPT-6.2 Astra against adversarial distillation.
Image: overview of TuringSpace — example tasks from the five categories, the generation pipeline and the model leaderboard (figure: Ding et al., via arXiv).
Sources: arXiv:2611.25387 — TuringSpace; arXiv — full text.
Transparency note: this article was produced with the assistance of artificial intelligence, cross-checking public posts on X and sources on the internet, and reviewed before publication. Check the original sources to confirm the information.
Por Muse