AutoBench Run 5 - December 2025

Latest AutoBench run with models Gpt 5.2, Claude Opus 4.5, Gemini 3 Flash and more

Latest

Date

December 19, 2025

Version

2025-12-19

Models

New Models

Run data

Overall Comparison Costs Latency P99 Latency Domain

AutoBench provides a complete dashboard with highly customizable domain-specific average ranks and precise efficiency metrics (average answer cost, average answer duration, P99 Latency). For further detail also check our Hugging Face Leaderboard.

Model	Score	Avg Cost ($ Cents)	Avg Latency (sec)	P99 Latency (sec)	Iterations
grok-4.1-fast	3.877365 (#30)	0.08 (#6)	24s (#2)	57s (#1)	312
gemini-2.5-flash-lite	3.94904 (#28)	0.21 (#11)	20s (#1)	69s (#2)	313
Nemotron-3-nano-30b-a3b	4.028261 (#25)	0.00 (#1)	30s (#3)	98s (#3)	314
Nova-2-lite-v1	4.059981 (#23)	3.94 (#30)	61s (#9)	132s (#4)	277
nova-premier-v1	3.473742 (#38)	1.30 (#25)	52s (#7)	135s (#5)	312
Gemini-3-flash-preview	4.303068 (#7)	1.95 (#27)	46s (#6)	137s (#6)	313
Mistral-medium-3.1	3.811798 (#33)	0.38 (#16)	52s (#8)	147s (#7)	306
Ministral-8b-2512	3.570151 (#36)	0.05 (#3)	31s (#4)	154s (#8)	306
gemini-2.5-flash	4.171935 (#15)	2.12 (#28)	66s (#10)	174s (#9)	312
Gpt-oss-20b	3.779105 (#35)	0.07 (#4)	39s (#5)	183s (#10)	310
gemini-3-pro-preview	4.405224 (#3)	6.85 (#32)	76s (#14)	186s (#11)	312
Mistral-large-2512	3.935105 (#29)	0.51 (#18)	90s (#19)	198s (#12)	307
grok-4.1-fast-thinking	4.206201 (#11)	0.27 (#12)	69s (#12)	207s (#13)	306
nemotron-nano-9b-v2	3.500291 (#37)	0.08 (#5)	67s (#11)	212s (#14)	311
gemini-2.5-pro	4.294065 (#9)	6.48 (#31)	87s (#18)	222s (#15)	313
Qwen3-next-80b-a3b-thinking	4.031744 (#24)	0.75 (#21)	78s (#16)	227s (#16)	312
llama-3.3-nemotron-super-49b-v1.5	3.783612 (#34)	0.18 (#9)	76s (#15)	240s (#17)	311
gpt-5-mini	4.287269 (#10)	0.91 (#22)	93s (#20)	258s (#18)	312
gpt-5-nano	4.060397 (#22)	0.34 (#15)	100s (#21)	269s (#19)	309
Olmo-3.1-32b-think	3.85021 (#32)	0.00 (#2)	122s (#24)	270s (#20)	307
gpt-oss-120b	4.181097 (#14)	0.11 (#8)	75s (#13)	292s (#21)	292
claude-haiku-4.5	4.170821 (#16)	3.79 (#29)	111s (#23)	317s (#22)	312
Kimi-K2-0905	4.107852 (#21)	0.33 (#14)	83s (#17)	329s (#23)	312
Qwen3-235B-A22B-2507	3.98095 (#27)	0.19 (#10)	105s (#22)	337s (#24)	302
Claude-opus-4.5	4.39496 (#4)	17.26 (#37)	144s (#28)	373s (#25)	313
Deepseek-v3.2	4.109586 (#20)	0.09 (#7)	125s (#25)	410s (#26)	311
GLM-4.5-Air	3.864646 (#31)	0.54 (#19)	163s (#29)	425s (#27)	306
Gpt-5.2	4.430061 (#2)	7.36 (#33)	130s (#26)	434s (#28)	312
Minimax-m2	3.990557 (#26)	0.71 (#20)	137s (#27)	473s (#29)	308
deepSeek-R1-0528	4.118577 (#19)	0.99 (#23)	171s (#31)	477s (#30)	308
claude-sonnet-4.5	4.30218 (#8)	11.39 (#36)	170s (#30)	477s (#31)	307
grok-4	4.197064 (#12)	8.12 (#34)	180s (#32)	562s (#32)	293
gpt-5.1	4.38364 (#5)	10.80 (#35)	227s (#34)	627s (#33)	310
GLM-4.6	4.132794 (#18)	1.25 (#24)	187s (#33)	630s (#34)	306
Kimi-K2-thinking	4.315342 (#6)	1.86 (#26)	248s (#35)	729s (#35)	287
Gpt-5.2-pro	4.476206 (#1)	81.88 (#38)	261s (#36)	784s (#36)	303
Qwen3-235B-A22B-Thinking-2507	4.196769 (#13)	0.32 (#13)	317s (#38)	811s (#37)	283
Deepseek-v3.2-speciale	4.141433 (#17)	0.47 (#17)	310s (#37)	833s (#38)	288