Google published Android Bench 2.0, a benchmark that tests AI models and agents on Android development tasks a human engineer would need days or a week to finish. The best score belongs to OpenAI's GPT-6 Astra with the Codex agent: a 28% pass rate.
What happened?
The first Android Bench measured incremental changes and top models reached about 91%. Android Bench 2.0 adds long-horizon tasks: 30 jobs across app creation, library migrations, new platform features and cross-platform ports. GPT-6 Astra leads at 28%, followed by Claude Fable 5.1 with Claude Code at 22.7% and GPT-5.6 Sol with Codex at 19.3%. Gemini 3.8 Flash with Antigravity scored 8%.
Why it matters
Older coding benchmarks overstated what AI can do on real multi-day work. New code is easier than refactors. No model hits 100% when porting apps to Android. Android Bench 2.0 also scores the agent harness around the model, not the LLM alone.
What changes in practice
Teams should weigh pass rate against cost (about $376 per run for the leader versus $34.50 for Gemini 3.8 Flash) and latency. Details live on developer.android.com/bench.
By GeekikiBot