Last updated Nov 20, 2026

ERNIE 5.5 Flash Preview Benchmarks, in Full: 20 Vendor Rows, 1 Win

ModelBrief · Published 2026-11-11 · Fact-check 2026-11-17 · Reference: Baidu official release page, aiho.net, aireiter, QbitAI, Stack Futures

The benchmark story of ERNIE 5.5 Flash Preview is one sentence: the official release page lists about 20 benchmark rows, all vendor-reported; 8 are cross-confirmed by multiple secondary sources, and only 1 of them is a win. This article gives the honest reading first, then tallies the tests one by one, and closes with three gaps: no independent scores, a CodeArena score not yet verified, and conflicting speed figures. Preview is a self-described unfinished intermediate build — every number on this table is intermediate, and the final version may change them.

The honest reading first: the selling point and the ceiling of 14B active

The 8 cross-confirmed benchmarks: Terminal-Bench 4.0 46.5, AutomationBench 61.9 (No. 1 in that test), DeepSWE v1.1 71.2, DRACO 84.9 (third), SpreadsheetBench V2 36.4, OfficeQA Pro V2 67.9, Agents' Last Exam (CLI) 29.0, APEX-Agents 1.1 64.2. Only 1 is a win (AutomationBench); the rest trail the flagships — TB 4.0 46.5 against Opus 5.5's 63.6 is a wide gap, DRACO 84.9 trails Hy4's 90.7, APEX-Agents 64.2 trails Hy4's 75.0. The correct reading is two sentences: first, a 14B-active model reaching this range is itself the selling point — only 14B of 280B total parameters activate per token (~5%), yet it sits close to the flagship tier on agent benchmarks; second, reading it as "beats the flagships" is a misreading — consistently trailing is the fact. Neither sentence works without the other.

The 8 cross-confirmed tests (vendor-reported)

Test (vendor-reported)ERNIE 5.5 Flash PreviewMain rivals (vendor chart, self-measured)
Terminal-Bench 4.046.5Claude Opus 5.5 63.6, GPT-6.1 Sol 56.1
AutomationBench61.9 (No. 1)Claude Opus 5.5 58.0, DeepSeek V4.1 Flash 57.2
DeepSWE v1.171.2Rival scores not cross-confirmed
DRACO84.9 (3rd)Hy4 90.7, Claude Opus 5.5 87.9
SpreadsheetBench V236.4Rival scores not cross-confirmed
OfficeQA Pro V267.9Rival scores not cross-confirmed
Agents' Last Exam (CLI)29.0Rival scores not cross-confirmed
APEX-Agents 1.164.2Hy4 75.0, Claude Opus 5.5 72.9

The vendor table has about 20 rows; the 8 above are the ones cross-confirmed by multiple secondary sources (aiho.net, todayforai, madechango); the remaining ~12 rows were not cross-confirmed, so this site does not list them. The AutomationBench 61.9 deserves its own note: the rivals are Claude Opus 5.5 (58.0) and DeepSeek V4.1 Flash (57.2), a lead of 3.9 and 4.7 points — the only No. 1 of the 8, and it should not be read as an overall lead.

Grouped comparison chart

Grouped benchmark comparison: ERNIE 5.5 Flash Preview, Claude Opus 5.5, Tencent Hy4 (vendor-reported)
Grouped comparison across four benchmarks (points; Baidu official release page, vendor-reported; rival scores as the vendor chart lists them, not independently measured; Hy4's TB 4.0 and AutomationBench figures were not verified in this research round, left blank)

The chart shows the four tests with head-to-head rival scores: TB 4.0, AutomationBench, DRACO and APEX-Agents 1.1. Orange is ERNIE 5.5 Flash Preview — its bar is the longest in only one of the four (AutomationBench); in the other three, the orange bar is shorter than the grey (Opus 5.5) and the teal (Hy4). Hy4's TB 4.0 and AutomationBench figures were not verified in this research round, so those slots are left blank rather than filled in.

CodeArena WebDev 1668: an AutoEval placeholder, the only third-party signal

CodeArena WebDev scores 1668 (±16), but on the Nov 11 board snapshot the row shows a blank rank, N/A votes and an AutoEval label — a placeholder used while human votes are still missing, not a ranking earned through battles; the license tag is Proprietary (weights not yet released, API-only). Arena said on Nov 12 that the model was "trending towards becoming the #2 open model", so "No. 2 open, No. 4 among Chinese models" is a forecast, not an established ranking. It is the only third-party board signal found. Its value is directional: a 14B-active model sitting in the same score band as the flagships corroborates the "14B active reaching the flagship range" reading of the vendor table — but until human votes validate it, it cannot be cited as an independent score.

No independent scores: AA, LiveBench and LMArena have no listing

As of November 16, 2026: Artificial Analysis has not listed ERNIE 5.5 Flash Preview (aireiter: no standalone Preview row found in the AA list; europesays on Nov 5 also reports no AA page), so there is no AA Index score; no LiveBench or LMArena listing was found; no independent body has reproduced any benchmark score. In other words, beyond the vendor table, the only number available is the press-reported CodeArena score. The tracking loop should watch for: an AA listing, first LiveBench / LMArena scores, and news of the final version (dropping the Preview suffix) — any one of them would change the credibility structure of this table.

Conflicting speed figures: vendor 200+ tok/s vs OpenRouter measured p50 75–79 tok/s

The vendor claims decoding at 200+ tokens/s; OpenRouter's measured median p50 is ~75–79 tok/s (Nov 12 data, free tier). The two figures differ by nearly 2× and come from different measurement bases (a vendor claim vs a third-party free-tier measurement); this site states both, takes no side, and adopts no single speed figure. Note what makes this conflict distinctive: it is not a versioning problem (like an index re-version) but a claim-vs-measurement problem — until a reproducible third-party measurement appears, 200+ can only be read as a vendor claim, not as a measured figure.

Demos are not benchmarks

The official demos (the 《云岫行》 game built in one 11-hour agent session, ~100 phone mini-games, airport/airliner Blender modeling) are not standardized benchmarks: all are end-to-end results inside Baidu's own agent framework, and a different harness or prompting strategy could change results by an order of magnitude; and the model is text-only, so it cannot see the screen. They show that the "execution layer for agent workloads" positioning has concrete examples behind it — they prove no benchmark score. madechango / 火星财经's Nov 13 hands-on verdict, "good taste, strong coding," is third-party hands-on reporting and should be read separately from the official demos.

FAQ

Can the ERNIE 5.5 Flash Preview benchmark scores be trusted?

Trust them cautiously. All ~20 rows on the official release page are vendor-reported, with no independent reproduction; as of November 16 there is no AA, LiveBench or LMArena listing; the only third-party signal, CodeArena WebDev 1668, is an AutoEval placeholder (zero human votes). Of the 8 cross-checked tests it wins only 1 (AutomationBench), and TB 4.0 46.5 trails the flagships clearly.

Why are only 8 tests cross-confirmed?

The vendor table has about 20 rows, but this research round could not read the official release page directly (a browser policy block), so facts were cross-checked against secondary sources that read it directly (digintoai, aiho.net); only 8 rows appear consistently across multiple secondary sources, and the remaining ~12 were not cross-confirmed — this site does not list them or invent them.

How fast is it really?

There is no single credible figure. The vendor claims 200+ tokens/s; OpenRouter's measured median p50 is ~75–79 tok/s (Nov 12 data, free tier). The two figures use different measurement bases, so this site adopts no single speed number — measure it yourself.

ModelBrief's take: the correct way to read this benchmark table is as "the ceiling of a lightweight execution layer," not as "a flagship challenger" — 14B active, AutomationBench No. 1, TB 4.0 46.5: those three numbers together are the true profile of ERNIE 5.5 Flash Preview. Wait for an AA listing, an independent reproduction, or the final version — any one of them is worth updating this article for; until then, this site will not present self-reported scores as independent conclusions.

Reference: Baidu official release page — ERNIE 5.5 Flash Preview · aiho.net — 280B deep dive · aireiter.com — API deep review (rewritten and organized by ModelBrief; facts per the official release page and reporting; the official page could not be read directly this round, facts cross-checked via secondary sources)