LongCat 2.5 Benchmarks: 31 Official Tests Reviewed

All 31 of LongCat 2.5's official benchmarks are Meituan's own measurements — the blog's evaluation table and the Hugging Face README table are identical, with rival scores half drawn from public reports and half run by Meituan itself. The highlights are two table-leading scores, BrowseComp 91.9 and 3DCodeBench 55.4, but same-priced DeepSeek V4.1 Flash leads on most coding rows; as of November 13, no independent body has scored 2.5. Here is the item-by-item breakdown.

31 benchmarks in three groups

The 31 split into Code Agent (10), General Agent (14) and Visual (7). Four scores lead their tables: BrowseComp 91.9, 3DCodeBench 55.4, ALE 38.5 and ResearchRubrics 81.0. The table below lists the headline scores, each with the 2.0 comparison and the main rivals; rival scores marked * come from public reports or leaderboards, unmarked ones were run by Meituan.

Also worth noting beyond the table: the three SWE Atlas scores (codebase Q&A 58.3, refactoring 55.2, test writing 47.8), SkillsBench 63.3, GDPVal-AA 84.8, FrontierScience 77.0, FinSearchComp T3 63.4, and on the visual side Vision2Web 78.6, CharXiv-RQ 90.2, MathVision 96.5 and PosterBench 77.4. Overall, 2.5's strengths concentrate in long-horizon agent and search tasks, with a gap to frontier closed models remaining on pure coding and visual Q&A.

Test conditions: how these numbers were produced

The shared conditions are thinking on, temperature 1.0, top_p 0.95. Terminal-Bench used the Claude Code harness v2.1.280 (max_seq 512K, v2.1 averaged over three runs); DeepSWE used mini-swe-agent (1M sequences, three-run average); BrowseComp used Meituan's own lightweight harness with only the search and browse_with_goal tools and a 1024-turn cap — so the BrowseComp 91.9 table lead should be discounted, as it is not the standard BrowseComp protocol. Rival scores marked * come from public reports or leaderboards; unmarked ones were run by Meituan and are officially "for reference only."

The same-price fight: DeepSeek V4.1 Flash

Both models share the same promo price ($0.30/$1.20), which makes the comparison the one that matters. On Meituan's table, DeepSeek leads more rows: TB 2.1 (90.6 vs 88.4), DeepSWE (74.2 vs 64.6), NL2Repo (65.4 vs 60.5), AutomationBench (54.8 vs 51.8), MCP Atlas (84.4 vs 83.1), SkillsBench (69.8 vs 63.3) and SWE Atlas refactoring (56.7 vs 55.2). LongCat leads in four places: TB 4.0 (38.5 vs 31.2), BrowseComp (91.9 vs 91.5), ProgramBench (25.5 vs 20.3) and SWE Atlas Q&A (58.3 vs 55.2); VideoMME has a published score only for LongCat (85.2).

The pattern is clear: DeepSeek wins on conventional coding tasks, LongCat fights back on the newer long-horizon benchmarks (TB 4.0, ProgramBench) and search tasks. Remember this is still Meituan's own table — DeepSeek's scores are marked *, drawn from public reports rather than a head-to-head run on the same harness.

The jump, the gap, and the stale comparison set

The 2.0-to-2.5 jump is the most credible part of the table: Terminal-Bench 4.0 rose from 3.0 to 38.5, DeepSWE from 9.7 to 64.6, ProgramBench from 2.5 to 25.5, HLE (w/ tools) from 37.5 to 63.1, AutomationBench from 15.8 to 51.8, and MCP Atlas from 61.8 to 83.1. The long-horizon RL investment shows up in these rows.

But independent measurement is still missing. Artificial Analysis has not listed 2.5 — its leaderboard carries only 2.0 and LongCat Flash Lite; ArtificialWatch published a "not rated yet" report on November 13, and llmperks shows "—" in its AA Index column. The calibration warning comes from 2.0: Artificial Analysis measured 2.0's Terminal-Bench 2.1 at 50.2%, versus 70.8 in Meituan's table — the same model, a different harness, a 20-point gap. Read 2.5's vendor scores with the same discount. The only community data point is one developer's self-run of TB 2.1 on the free preview at 48.7% (single person, preview version, different harness) — reference only, not a 2.5 score.

The comparison set is also dated: the official dashboard omits Opus 5.5, GPT-6.1 Sol and Kimi K3.1 Pro, a point both eesel and Hacker News raised. When reading this table, the absent rivals matter as much as the ones present.