Premium AI Model Tests — The Flagships on Genuinely Hard Tasks
Our cheap-model tests answer “what's the best value LLM?” This page asks the opposite question: when you reach for the most capable models money can buy, what do the extra dollars and seconds actually buy you? We took the four high-end flagships — GPT-5.4 Pro, Claude Opus 4.6, Gemini 3.1 Pro and GLM 5.2 in Max reasoning mode — and sent each the same deliberately hard prompts through the live All AI Ask API. Every number below is real, and both tasks were verified programmatically, not just eyeballed.
Raw dataset: data.json. Cite this: All AI Ask Premium Model Test Evidence Dataset, retrieved 2026-06-21T00:00:00.000Z.
Which flagship model leads the premium-model tests?
Claude Opus 4.8 tops the premium-model leaderboard at 100 average accuracy across 2 hard tasks and 8 live runs covering 4 flagship models from 4 providers. Every task ran identical prompts through the live API with programmatically verified grading; per-task tables, verbatim outputs, and the evidence dataset are linked below.
The tasks
Solving a Constraint Logic Puzzle
A five-house constraint-satisfaction puzzle with five interlocking clues and a single valid solution. The model must reason through both cases, eliminate the dead end, and report the exact arrangement and count.
Median of Two Sorted Arrays in O(log n)
A classic hard algorithm: compute the median of two sorted lists in O(log(min(m,n))) time. A merge is explicitly disallowed, so the model must implement the tricky binary-search partition correctly — including empty-list and even/odd edge cases — and return code only.
Overall leaderboard
Averaged across both hard tasks. Accuracy is graded against each task's published criteria and cross-checked programmatically. Ties on accuracy break toward the cheaper model.
| # | Model | Avg accuracy | Avg speed | List price | Total cost |
|---|---|---|---|---|---|
| 🥇 | Claude Opus 4.8Anthropic | 100 | 99.1 t/s | $25/M | $0.036505 |
| 2 | GPT-5.4 ProOpenAI | 100 | 12.2 t/s | $180/M | $0.2565 |
| 3 | GLM 5.2 (Max)Z.ai | 99.5 | 59.4 t/s | $4.4/M | $0.02237 |
| 4 | Gemini 3.1 ProGoogle | 98.5 | 20.5 t/s | $12/M | $0.010744 |
How we tested
- The cohort: the four high-end flagships — GPT-5.4 Pro, Claude Opus 4.6, Gemini 3.1 Pro, and GLM 5.2 run in Max reasoning mode.
- Identical prompts: each model received the same prompt through the live API; GLM 5.2 used
reasoningEffort: "max", the others their default flagship settings. - Real metrics: latency, token counts, and cost come straight from the API for each run. (GPT-5.4 Pro's token count is estimated from output length — the API under-reports it for that model.)
- Verified accuracy: the logic puzzle was checked by exhaustive search and every code answer was executed against a 5,000-case correctness harness — so the grades aren't guesswork.