AutoBench Run 6 - August 2026

Twenty models scored on 252 questions across ten subject domains. Score, cost, and latency are the per-answer committee rank, billed cost, and generation time. Routers in the AutoBench Router Benchmark are scored against this answer matrix.

Latest

Date

August 29, 2026

Version

2026-08-29

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

Track the cost efficiency of LLM models across 10 specialized domains. Costs are measured in cents per response, helping you identify the most economical models for your specific use case.

Model	Average (All Topics)	Computer Science	Philosophy	Social Science	Language	Science	Technology	Arts	Literature	History	Agentic
GPT-5.6 Sol	7.87 (#20)	22.75 (#20)	4.98 (#20)	9.83 (#20)	8.01 (#20)	9.50 (#20)	9.11 (#20)	7.48 (#20)	3.83 (#20)	3.74 (#20)	1.92 (#20)
Claude Sonnet 5	4.05 (#19)	10.00 (#19)	3.04 (#19)	3.55 (#19)	4.46 (#19)	8.87 (#19)	4.59 (#19)	2.20 (#18)	2.19 (#19)	1.48 (#19)	0.90 (#19)
Kimi K2.6	1.45 (#16)	3.23 (#17)	1.12 (#17)	1.24 (#16)	2.00 (#17)	1.40 (#14)	2.44 (#18)	1.46 (#16)	0.90 (#16)	0.76 (#17)	0.69 (#18)
Qwen3.8 Max	1.63 (#17)	2.74 (#16)	1.29 (#18)	1.39 (#17)	2.29 (#18)	2.06 (#16)	2.15 (#17)	2.47 (#19)	1.01 (#17)	0.75 (#16)	0.49 (#15)
Grok 4.6	1.64 (#18)	3.75 (#18)	0.94 (#16)	1.43 (#18)	1.65 (#16)	2.36 (#17)	1.81 (#16)	1.88 (#17)	1.17 (#18)	1.07 (#18)	0.61 (#16)
Gemini 3.7 Flash	1.07 (#14)	2.47 (#15)	0.54 (#14)	0.75 (#14)	1.25 (#15)	1.85 (#15)	1.46 (#15)	1.06 (#15)	0.67 (#14)	0.52 (#14)	0.27 (#13)
Muse Spark 1.2	1.15 (#15)	1.62 (#13)	0.84 (#15)	0.91 (#15)	1.11 (#14)	2.38 (#18)	1.25 (#14)	0.99 (#14)	0.82 (#15)	0.74 (#15)	0.69 (#17)
Nemotron 3 Ultra	0.69 (#13)	1.73 (#14)	0.48 (#13)	0.52 (#12)	1.08 (#13)	0.76 (#11)	1.11 (#13)	0.75 (#13)	0.24 (#9)	0.21 (#10)	0.21 (#12)
GPT-5.6 Luna	0.43 (#8)	0.90 (#8)	0.37 (#11)	0.48 (#11)	0.47 (#9)	0.47 (#8)	0.64 (#12)	0.54 (#10)	0.32 (#11)	0.21 (#9)	0.12 (#10)
Grok 4.3	0.50 (#10)	0.95 (#9)	0.30 (#9)	0.39 (#10)	0.54 (#10)	0.97 (#12)	0.57 (#11)	0.36 (#8)	0.34 (#12)	0.26 (#12)	0.32 (#14)
GLM 5.3	0.57 (#12)	1.17 (#12)	0.45 (#12)	0.71 (#13)	0.84 (#11)	0.61 (#9)	0.56 (#9)	0.50 (#9)	0.37 (#13)	0.35 (#13)	0.18 (#11)
GLM 5.2	0.51 (#11)	1.12 (#11)	0.28 (#8)	0.29 (#8)	0.85 (#12)	0.99 (#13)	0.56 (#10)	0.59 (#11)	0.18 (#8)	0.15 (#8)	0.11 (#8)
MiniMax M3	0.44 (#9)	1.01 (#10)	0.34 (#10)	0.37 (#9)	0.44 (#8)	0.69 (#10)	0.39 (#8)	0.65 (#12)	0.28 (#10)	0.25 (#11)	0.12 (#9)
Nemotron 3.5 Lightning	0.15 (#7)	0.33 (#7)	0.05 (#4)	0.10 (#6)	0.21 (#7)	0.28 (#7)	0.22 (#7)	0.10 (#6)	0.06 (#5)	0.05 (#4)	0.09 (#7)
Muse Glimmer 30B	0.11 (#6)	0.16 (#5)	0.09 (#7)	0.10 (#7)	0.13 (#5)	0.18 (#6)	0.13 (#6)	0.10 (#5)	0.07 (#6)	0.07 (#6)	0.06 (#5)
Qwen3.5-9B	0.10 (#5)	0.13 (#4)	0.09 (#6)	0.08 (#5)	0.14 (#6)	0.13 (#4)	0.13 (#4)	0.11 (#7)	0.09 (#7)	0.08 (#7)	0.06 (#6)
Hy3	0.09 (#4)	0.18 (#6)	0.08 (#5)	0.08 (#4)	0.10 (#4)	0.16 (#5)	0.13 (#5)	0.09 (#4)	0.05 (#4)	0.05 (#5)	0.04 (#4)
DeepSeek V4 Flash 0731	0.05 (#3)	0.12 (#3)	0.04 (#3)	0.04 (#3)	0.07 (#3)	0.08 (#3)	0.06 (#3)	0.06 (#3)	0.02 (#3)	0.03 (#3)	0.02 (#3)
Gemma 4 31B IT	0.02 (#2)	0.04 (#2)	0.02 (#1)	0.02 (#1)	0.02 (#1)	0.03 (#2)	0.03 (#2)	0.02 (#1)	0.02 (#1)	0.02 (#1)	0.01 (#2)
Gpt oss 120b	0.02 (#1)	0.03 (#1)	0.02 (#2)	0.03 (#2)	0.02 (#2)	0.03 (#1)	0.03 (#1)	0.02 (#2)	0.02 (#2)	0.02 (#2)	0.01 (#1)