Cat Board
A self-funded, independently run community benchmark — not official, not run by Oriel. The question set is private and rotates monthly, so this is one narrow view of long-term trends, not an authoritative or complete ranking. Don't take it as gospel.
Source: llm2014/llm_benchmark · Original site
This edition 2026-09
1
GPT-6 Astra (xhigh)2.76%
GPT-6 Astra (xhigh)2.76%2
Fable 5.1 (xhigh ~Estimated)3.82%
Fable 5.1 (xhigh ~Estimated)3.82%3Opus 5.5 (xhigh ~Estimated)3.17%
Opus 5.5 (xhigh ~Estimated)3.17%4GPT-5.6 Sol (xhigh)13.36%
GPT-5.6 Sol (xhigh)13.36%5GPT-6 Sol (xhigh)16.33%
GPT-6 Sol (xhigh)16.33%6Opus 5 (xhigh)10.63%
Opus 5 (xhigh)10.63%7
Seed-2.1-pro 0915 (high)16.89%
Seed-2.1-pro 0915 (high)16.89%8
GLM-5.3 (max)17.46%
GLM-5.3 (max)17.46%9Opus 5.5 (xhigh w/ refuse)3.79%
Opus 5.5 (xhigh w/ refuse)3.79%10
Kimi-K3 (max)10.52%
Kimi-K3 (max)10.52%11
Grok 4.7 (xhigh)11.62%
Grok 4.7 (xhigh)11.62%12
DeepSeek V4.1 Flash (max)10.47%
DeepSeek V4.1 Flash (max)10.47%13
Gemini 3.8 Flash (high)11.37%
Gemini 3.8 Flash (high)11.37%14Gemini 3.1 Pro (high)17.82%
Gemini 3.1 Pro (high)17.82%15
Muse Spark 1.3 (max)18.47%
Muse Spark 1.3 (max)18.47%5095Gap
Top 15 of 56 by peak score
| Model | Peak score | Median score | Peak–median gap | Change | Avg. time (s) | Tokens | Cost (CNY) | Price (CNY/1M) | Released | Reasoning |
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra (xhigh) | 91.27 | 88.75 | 2.76% | +25.1% | 241 | 12198 | ¥117.83 | ¥345.00 | 26-09-05 | On |
| Fable 5.1 (xhigh ~Estimated) | 84.56 | 81.33 | 3.82% | - | 481 | 41434 | ¥400.25 | ¥345.00 | 26-09-02 | On |
| Opus 5.5 (xhigh ~Estimated) | 82.99 | 80.36 | 3.17% | +26.3% | 295 | 33096 | ¥127.88 | ¥138.00 | 26-09-23 | On |
| GPT-5.6 Sol (xhigh) | 78.90 | 68.36 | 13.36% | - | 478 | 21006 | ¥81.17 | ¥138.00 | 26-07-10 | On |
| Opus 5.5 (xhigh w/ refuse) | 69.48 | 66.85 | 3.79% | - | 295 | 33096 | ¥127.88 | ¥138.00 | 26-09-23 | On |
| Opus 5 (xhigh) | 71.20 | 63.63 | 10.63% | - | 500 | 29868 | ¥144.26 | ¥172.50 | 26-07-25 | On |
| Fable 5.1 (xhigh w/ refuse) | 66.70 | 63.47 | 4.84% | - | 481 | 41434 | ¥400.25 | ¥345.00 | 26-09-02 | On |
| GPT-6 Sol (xhigh) | 74.03 | 61.94 | 16.33% | - | 220 | 18451 | ¥35.65 | ¥69.00 | 26-09-23 | On |
| Kimi-K3 (max) | 69.13 | 61.86 | 10.52% | +32.6% | 1241 | 38577 | ¥113.42 | ¥105.00 | 26-07-16 | On |
| DeepSeek V4.1 Flash (max) | 68.74 | 61.54 | 10.47% | +18.9% | 584 | 85949 | ¥19.25 | ¥8.00 | 26-09-10 | On |
| Grok 4.7 (xhigh) | 68.99 | 60.97 | 11.62% | +21.9% | 1421 | 79365 | ¥73.60 | ¥33.12 | 26-09-22 | On |
| Gemini 3.8 Flash (high) | 68.08 | 60.34 | 11.37% | +9.4% | 395 | 39333 | ¥28.50 | ¥25.88 | 26-09-03 | On |
| Seed-2.1-pro 0915 (high) | 70.12 | 58.28 | 16.89% | +44.8% | 1200 | 43884 | ¥36.86 | ¥30.00 | 26-09-16 | On |
| GLM-5.3 (max) | 69.69 | 57.52 | 17.46% | +16.8% | 1187 | 64363 | ¥50.46 | ¥28.00 | 26-08-19 | On |
| Gemini 3.1 Pro (high) | 67.50 | 55.47 | 17.82% | +5.8% | 238 | 23891 | ¥55.39 | ¥82.80 | 26-02-19 | On |
| GLM-5.3-Flash (max) | 64.06 | 55.16 | 13.89% | - | 1025 | 44240 | ¥3.47 | ¥2.80 | 26-08-26 | On |
| Muse Spark 1.3 (max) | 66.72 | 54.40 | 18.47% | +38% | 439 | 66822 | ¥54.87 | ¥29.33 | 26-09-02 | On |
| DeepSeek V4 Pro 0813 (max) | 64.73 | 53.70 | 17.04% | +26.4% | 1643 | 60698 | ¥45.89 | ¥27.00 | 26-08-13 | On |
 Qwen3.8-Max (xhigh) | 59.80 | 53.55 | 10.45% | - | 1419 | 70320 | ¥70.88 | ¥36.00 | 26-08-03 | On |
| DeepSeek V4.1 Flash (high) | 63.26 | 52.09 | 17.66% | - | 640 | 68561 | ¥15.36 | ¥8.00 | 26-09-10 | On |
| Grok 4.6 (high) | 64.54 | 49.99 | 22.54% | +5.7% | 834 | 36510 | ¥42.32 | ¥41.40 | 26-08-13 | On |
 Step 5 Preview (high) | 62.80 | 49.87 | 20.59% | +413.8% | 1224 | 52901 | ¥29.62 | ¥20.00 | 26-09-20 | On |
| Qwen3.8-Flash (xhigh) | 59.09 | 48.14 | 18.53% | - | 897 | 62634 | ¥4.74 | ¥2.70 | 26-08-26 | On |
| GPT-5.6 Luna (xhigh) | 59.93 | 44.29 | 26.10% | - | 308 | 39417 | ¥9.14 | ¥8.28 | 26-07-10 | On |
| Seed-2.1-pro (high) | 50.78 | 40.23 | 20.78% | +7.7% | 2139 | 69582 | ¥58.45 | ¥30.00 | 26-06-23 | On |
| Gemini 3.7 Flash (low) | 48.83 | 39.72 | 18.66% | - | 57 | 9377 | ¥6.79 | ¥25.88 | 26-08-13 | On |
| Qwen3.8-27B (xhigh) | 50.53 | 39.48 | 21.87% | - | 2180 | 71692 | ¥24.09 | ¥12.00 | 26-08-14 | On |
| Muse Spark 1.2 (xhigh) | 51.64 | 39.39 | 23.72% | - | 340 | 39800 | ¥32.68 | ¥29.33 | 26-08-05 | On |
 Hy3 (high) | 46.52 | 37.69 | 18.98% | +23.2% | 1113 | 46845 | ¥5.25 | ¥4.00 | 26-07-06 | On |
 MiniMax-M3 | 38.42 | 33.16 | 13.69% | +33.3% | 1030 | 67827 | ¥15.95 | ¥8.40 | 26-06-01 | On |
| Qwen3.7-Plus (high) | 43.41 | 31.59 | 27.23% | +24.9% | 1227 | 57472 | ¥12.87 | ¥8.00 | 26-06-02 | On |
| Sonnet 5 (xhigh) | 40.34 | 31.55 | 21.79% | - | 645 | 51921 | ¥100.31 | ¥69.00 | 26-07-01 | On |
| GPT-6 Luna (xhigh) | 37.78 | 25.55 | 32.37% | - | 281 | 26202 | ¥2.53 | ¥3.45 | 26-09-23 | On |
| Seed-2.0-lite 0428 (high) | 29.80 | 22.52 | 24.43% | - | 639 | 23332 | ¥2.35 | ¥3.60 | 26-05-07 | On |
| Opus 5 | 30.55 | 20.59 | 32.60% | +14% | 201 | 13862 | ¥66.95 | ¥172.50 | 26-07-25 | Standard |
| Gemini 3.5 Flash Lite (high) | 26.71 | 19.76 | 26.02% | +27.1% | 185 | 16583 | ¥8.01 | ¥17.25 | 26-07-21 | On |
Spark X2.5 | 32.45 | 17.73 | 45.36% | +317% | 645 | 45239 | ¥7.60 | ¥6.00 | 26-09-06 | On |
 Ling-3.0-flash | 22.39 | 13.47 | 39.84% | - | 727 | 82850 | - | - | 26-07-24 | On |
  MiMo-V2.5-Pro | 24.84 | 12.52 | 49.60% | +18.9% | 631 | 30724 | ¥5.16 | ¥6.00 | 26-04-23 | On |
| Gemma 4 31B | 15.38 | 12.51 | 18.66% | - | 706 | 14569 | ¥1.13 | ¥2.76 | 26-04-03 | On |
| GPT-5.5 Instant | 18.71 | 12.11 | 35.28% | - | 29 | 1599 | ¥9.27 | ¥207.00 | 26-05-06 | Standard |
| Step-3.7-Flash | 23.37 | 12.05 | 48.44% | - | 317 | 45857 | ¥10.40 | ¥8.10 | 26-05-29 | On |
openPangu-2.0-Pro | 15.15 | 11.19 | 26.14% | - | 707 | 29159 | ¥11.84 | ¥14.50 | 26-07-31 | On |
| Qwen3.7-Plus | 14.94 | 10.81 | 27.64% | - | 305 | 8554 | ¥1.92 | ¥8.00 | 26-06-02 | Standard |
 Mistral Medium 3.5 | 13.11 | 10.61 | 19.07% | - | 206 | 26507 | ¥38.41 | ¥51.75 | 26-04-30 | On |
Dots3-Note Preview | 15.82 | 10.32 | 34.77% | - | 218 | 23936 | - | - | 26-08-14 | On |
| DeepSeek V4.1 Flash | 13.94 | 9.96 | 28.55% | +56.6% | 29 | 3838 | ¥0.86 | ¥8.00 | 26-09-10 | Standard |
openPangu-2.0-Flash | 15.58 | 9.05 | 41.91% | - | 593 | 27215 | ¥1.22 | ¥1.60 | 26-06-30 | On |
| Qwen3.8-27B | 12.23 | 8.90 | 27.23% | - | 194 | 8565 | ¥2.88 | ¥12.00 | 26-08-14 | Standard |
 ERNIE 5.1 | 13.07 | 7.62 | 41.70% | +2.4% | 479 | 23797 | ¥11.99 | ¥18.00 | 26-05-09 | On |
| Gemini 3.1 Flash Lite | 8.71 | 7.26 | 16.65% | - | 11 | 712 | ¥0.21 | ¥10.35 | 26-03-03 | Standard |
| Sonnet 5 | 12.94 | 6.57 | 49.23% | +21% | 124 | 7196 | ¥13.90 | ¥69.00 | 26-07-01 | Standard |
 LongCat-2.0 | 13.58 | 6.52 | 51.99% | - | 379 | 13041 | ¥2.92 | ¥8.00 | 26-06-30 | On |
| Seed-2.1-pro 0915 | 11.50 | 6.23 | 45.83% | +41.2% | 38 | 1317 | ¥1.11 | ¥30.00 | 26-09-16 | Standard |
| Seed-2.1-pro | 13.35 | 4.74 | 64.49% | - | 338 | 8751 | ¥7.35 | ¥30.00 | 26-06-23 | Standard |
| Mistral Medium 3.5 | 6.06 | 4.41 | 27.23% | - | 17 | 2069 | ¥3.00 | ¥51.75 | 26-04-30 | Standard |
The question set rotates monthly; only the latest edition is shown. Scores are not comparable across months, so this site does not chart a trend for them.
This edition 2026-09
| Model | Simple Model(H) | iOS+Server(I) | Animation(J) | Data Process(K) | Metal(L) | Science(M) | Unprompted | Scaffold | Reasoning |
|---|---|---|---|---|---|---|---|---|---|
| Opus 5.5 (max) | Pass | Pass | 4/A+(341.72) | Perfect(211.83) | Pending | 2/A+(1365.55) | 5 | Claude Code | On |
| GPT-6 Astra (max) | 5/A+(41.53) | 3/A+(64.74) | 4/A+(75.84) | 5/A(61.32) | 8/B+(114.59) | 10/A(91.22) | 4 | Codex CLI | On |
| Fable 5 (high) | 2/A+(90.52) | 3/A+(103.95) | 7/A(59.02) | Skipped | Skipped | Skipped | 1 | Claude Code | On |
| Opus 5 (max) | 4/A(123.94) | 1/A+(122.57) | 12/B+(573.18) | 16/D(304.84) | 18/C(233.48) | 15/B(990.08) | 4 | Claude Code | On |
| GPT-6 Sol (max) | Pass | Pass | 16/B(27.74) | 12/C(90.31) | Skipped | 14/B(42.82) | 1 | Codex CLI | On |
| GPT-5.6 Sol (max) | 4/A(49.32) | 8/A(20.08) | 23/B(75.54) | 8/B(607.30) | 13/C(98.19) | Skipped | 3 | Codex App | On |
| Kimi-K3 (max) | 6/A(29.59) | 5/A(43.22) | 22/B(43.23) | 28/D+(383.49) | 25/D(193.42) | 26/C+(267.49) | 2 | Claude Code | On |
| Grok 4.7 (high) | 8/B(53.87) | 3/A+(15.68) | 14/C+(32.91) | 12/C+(139.47) | Skipped | Skipped | 0 | Grok Build | On |
| GLM-5.3 (max) | 8/B+(11.48) | 7/A(25.25) | 20/C+(90.76) | 30/D(83.29) | 24/D(343.68) | Failed | 0 | Claude Code | On |
| Qwen3.8-Max-0902 (max) | 8/B(28.26) | 7/A(109.76) | 21/C(102.71) | 13/D(211.18) | Pending | 26/D(347.42) | 0 | Claude Code | On |
| Gemini 3.7 Flash (high) | 14/B(9.38) | 8/B(16.84) | 14/B(14.28) | 34/D(28.93) | Skipped | Skipped | 4 | OpenCode CLI | On |
| MiMo-V2.6-Pro (high) | 16/C(10.11) | 5/B+(1.55) | 14/B(7.23) | Failed | Skipped | Failed | 0 | Claude Code | On |
| Grok 4.6 (high) | 16/C+(17.67) | 7/B+(18.69) | 20/C+(23.19) | Skipped | Skipped | Skipped | 1 | Grok Build | On |
| Seed-2.1-Pro 0915 (high) | 11/B(20.89) | 5/A(50.68) | Failed | 30/D(71.77) | Pending | Pending | 0 | Claude Code | On |
| DeepSeek V4.1 Flash (max) | 12/B(2.79) | 7/B(2.92) | Failed | 20/C(5.88) | Pending | Skipped | 0 | Claude Code | On |
| GLM-5.3-Flash (max) | 9/B(4.21) | 13/B(6.63) | Failed | Skipped | Skipped | Skipped | 0 | Claude Code | On |
| DeepSeek V4 Pro 0813 (max) | 14/B(13.46) | 16/C(12.73) | Failed | 16/B(18.64) | 27/D(69.58) | Skipped | 0 | Claude Code | On |
| Muse Spark 1.2 (xhigh) | 26/C(15.82) | 11/B(17.76) | Failed | Skipped | Skipped | Skipped | 0 | Claude Code | On |
| DeepSeek V4 Flash 0731 (max) | 17/B(3.95) | 24/D+(2.72) | Failed | Failed | Skipped | Skipped | 2 | Claude Code | On |
| Sonnet 5 (high) | 22/C(154.97) | 16/C+(72.34) | Failed | Skipped | Skipped | Skipped | 1 | Claude Code | On |
| Hy3 (high) | 10/B(2.01) | 20/C+(13.29) | Failed | Failed | Skipped | Skipped | 0 | Claude Code | On |
| GPT-5.6 Luna (max) | 27/C(3.84) | 21/C(4.90) | Failed | Skipped | Skipped | Skipped | 0 | Codex | On |
| MiniMax-M3 | 30/D(2.42) | 17/C+(8.28) | Failed | Skipped | Skipped | Skipped | 1 | Claude Code | On |
The code methodology was revised to v3: no longer a single score, but per-project Pass / Skip / Pending / Failed, plus a "correction rounds / final grade" pair. The question set rotates monthly; only the latest edition is shown. Scores are not comparable across months, so this site does not chart a trend for them.
This edition 2025-11This category hasn't been updated in 10 months
1Gemini 3 Pro4.69%
Gemini 3 Pro4.69%2Gemini 2.5 Pro3.58%
Gemini 2.5 Pro3.58%3GPT-5 (high)12.86%
GPT-5 (high)12.86%4Seed-1.6-vision (Think)7.82%
Seed-1.6-vision (Think)7.82%5qwen3-vl-235b-a22b (Think)8.10%
qwen3-vl-235b-a22b (Think)8.10%6Seed-1.6-thinking-07158.15%
Seed-1.6-thinking-07158.15%7GLM-4.5V7.22%
GLM-4.5V7.22%8Seed-1.6-vision10.53%
Seed-1.6-vision10.53%9Step-37.19%
Step-37.19%10Qwen3-vl-235b-a22b4.72%
Qwen3-vl-235b-a22b4.72%11Sonnet 4 (Think)18.37%
Sonnet 4 (Think)18.37%12Sonnet 49.48%
Sonnet 49.48%13ERNIE 4.5 Turbo VL 082911.74%
ERNIE 4.5 Turbo VL 082911.74%14Qwen-vl-max-2025-08-139.83%
Qwen-vl-max-2025-08-139.83%15hunyuan-t1-vision 061912.02%
hunyuan-t1-vision 061912.02%3080Gap
Top 15 of 20 by peak score
| Model | Peak score | Median score | Peak–median gap | Price (CNY/1M) | Avg. tokens | Cost (CNY) | Avg. time (s) | Released | Reasoning |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3 Pro | 73.94 | 70.47 | 4.69% | ¥86.4 | 10039 | ¥16.48 | 138 | 25-11-19 | On |
| Gemini 2.5 Pro | 64.09 | 61.80 | 3.58% | ¥72.0 | 7277 | ¥9.95 | 79 | 25-05-06 | On |
| GPT-5 (high) | 61.73 | 53.79 | 12.86% | ¥72.0 | 6271 | ¥8.58 | 182 | 25-08-07 | On |
| Seed-1.6-vision (Think) | 60.58 | 55.85 | 7.82% | ¥8.0 | 6695 | ¥1.02 | 83 | 25-08-15 | On |
| qwen3-vl-235b-a22b (Think) | 58.21 | 53.50 | 8.10% | ¥20.0 | 5031 | ¥1.91 | 101 | 25-09-24 | Standard |
| Seed-1.6-thinking-0715 | 55.39 | 50.87 | 8.15% | ¥8.0 | 6104 | ¥0.93 | 118 | 25-07-14 | On |
| GLM-4.5V | 53.26 | 49.42 | 7.22% | ¥6.0 | 2814 | ¥0.32 | 51 | 25-08-11 | On |
| Seed-1.6-vision | 48.89 | 43.74 | 10.53% | ¥8.0 | 2263 | ¥0.34 | 29 | 25-08-15 | Standard |
| Step-3 | 48.85 | 45.34 | 7.19% | ¥8.0 | 6033 | ¥0.92 | 294 | 25-07-31 | On |
| Qwen3-vl-235b-a22b | 47.88 | 45.62 | 4.72% | ¥8.0 | 1590 | ¥0.24 | 34 | 25-09-24 | Standard |
| Sonnet 4 (Think) | 47.57 | 38.83 | 18.37% | ¥108.0 | 2895 | ¥5.94 | 42 | 25-05-23 | On |
| Sonnet 4 | 45.53 | 41.21 | 9.48% | ¥108.0 | 931 | ¥1.91 | 17 | 25-05-23 | Standard |
| ERNIE 4.5 Turbo VL 0829 | 45.31 | 39.99 | 11.74% | ¥9.0 | 3433 | ¥0.59 | 245 | 25-08-29 | On |
| Qwen-vl-max-2025-08-13 | 44.36 | 40.00 | 9.83% | ¥4.0 | 2520 | ¥0.19 | 115 | 25-08-13 | Standard |
| hunyuan-t1-vision 0619 | 42.68 | 37.55 | 12.02% | ¥9.0 | 6009 | ¥1.03 | 115 | 25-06-19 | On |
SenseNova-V6-Reasoner | 40.71 | 37.38 | 8.18% | ¥16.0 | 2316 | ¥0.70 | 90 | 25-04-10 | On |
| Seed-1.6 | 39.30 | 35.32 | 10.15% | ¥8.0 | 1006 | ¥0.15 | 23 | 25-06-11 | Standard |
| hunyuan-turbos-vision 0619 | 38.05 | 34.54 | 9.23% | ¥9.0 | 1188 | ¥0.20 | 31 | 25-06-19 | Standard |
SenseNova-V6-Pro | 34.41 | 26.68 | 22.47% | ¥9.0 | 689 | ¥0.12 | 18 | 25-04-10 | Standard |
| MiniMax-Text-01 | 25.17 | 23.38 | 7.10% | ¥8.0 | 587 | ¥0.09 | 24 | 25-01-13 | Standard |
The question set rotates monthly; only the latest edition is shown. Scores are not comparable across months, so this site does not chart a trend for them.