Updated
Updated · 9to5Google · Jul 8
Google Recasts Android Bench on Harbor as Claude Fable 5 Leads at 84.5
Updated
Updated · 9to5Google · Jul 8

Google Recasts Android Bench on Harbor as Claude Fable 5 Leads at 84.5

3 articles · Updated · 9to5Google · Jul 8

Summary

  • Google refreshed Android Bench under the Harbor framework, and Claude Fable 5 took the top spot with an 84.5 score, 4.3 points ahead of GPT-5.5 at 80.2.
  • Harbor replaces the mini-swe-agent v1 setup Google had used, standardizing Android coding tests and letting developers analyze models against specific use cases.
  • Google is also opening Android Bench to community input, inviting developers to submit Android tasks and share benchmark evaluations that will help shape the dataset.
  • The reranking reshuffled most prior scores because the underlying rules changed; Gemini 3.1 Pro Preview placed fifth at 73.7, while the leaderboard now also shows latency and cost.

Insights

Google's AI ranks fifth on its own benchmark. Is the top-ranked model actually worth the cost for developers?
As AI models master Android code, can we trust benchmarks that might just be testing their memory?
Beyond accuracy, what hidden costs and efficiency trade-offs are AI coding assistants hiding from Android developers?