Muse Ember's benchmark data comes in two layers: the top layer is the Intelligence Index of 36 (max effort), independently published by Artificial Analysis on November 9; the lower layer is six component scores reproduced in charts in Meta's launch post. The two layers carry different weight — this article separates who is speaking first, then takes each score apart, and collects the weak spots at the end.
AA Intelligence Index v5.0 (November 9, 2026, max effort): Ember 36, level with Claude Haiku 5.5 (max), behind Muse Spark 1.3 (max) at 37. Cost per task: Ember $0.14, Haiku $0.41, Spark $2.39. AA's own characterisation is the "cheapest model scoring 36 or higher"; at-pulse adds that Ember ranks 2nd of 33 comparable models at ~$0.14 per task. This is the only number in Ember's entire benchmark record that an independent board published itself — and it is the source of the headline "ties Haiku at a third of the cost".
Meta's launch-post charts (all vendor-reported, reproducing AA evaluations; rival scores as Meta lists them, not each vendor's published values): AutomationBench-AA 37.8% (Haiku 13.9%, Luna 20.2%), the widest gap — Ember scores nearly 2.7× Haiku; Terminal-Bench 4.0 40.9% (Haiku 39.2%), a win by just 1.7 points; FrontierCode 2.0-AA 27.2% (Haiku 34.9%), a loss by 7.7 points; OSWorld 2.1 69.5% (Haiku 68.9%, Luna 50.5%), a win by just 0.6 points; Agents' Last Exam 61.6 (57.0 / 57.1); PerceptionBench 80.6 (78.6 / 62.8). Ember wins five of six — but two of the wins are under 2 points, and the loss is 7.7 points. Beyond the headline, the real distribution is "a big business-workflow win, a clear coding loss, and narrow wins elsewhere".
AA-Omniscience 21.1%, the lowest among models with similar overall index scores (via alphasignal). Index 36 summarises general intelligence; Omniscience measures a different dimension — Ember's 36 hides a bottom-of-class result. Read together with the FrontierCode loss, Ember's capability profile is complete: level with Haiku on the headline index, with clear weak spots in coding and on Omniscience. When choosing a model, don't just remember 36 — remember 21.1% alongside it.
Ridge's caution deserves its own paragraph: the AA Coding Agent Index of 46 is a figure from Meta's launch post, and Ridge notes that "Meta's other figures … are its claims". In other words, 46 has the same provenance as the six component benchmarks — Meta. Don't cite it as a second independently published AA score. As of Nov 16, the only Ember results AA published independently are the Index of 36 and, via reporting, Omniscience at 21.1%.
Meta's headline claim is "more intelligence per dollar than Claude Haiku 5.5 or GPT-6 Luna at any reasoning setting". Three caveats: first, it is Meta's reading of AA results, not AA's conclusion; second, the unit economics of the cost comparison have not been third-party audited (the dineroit / codema caution); third, Meta's chart notes Ember's costs were estimated at proposed pricing. Most importantly, Meta published benchmark results only at max effort — the "any reasoning setting" claim has no lower-effort data behind it. Simon Willison's hands-on test (Nov 10) found quality does not rise steadily with effort: one request at high failed to return a complete drawing. A single-person, single-task observation, not a benchmark — but it directly questions the "any setting" wording.
As of Nov 16, LiveBench has no Ember run and LMArena has no listing; Simon Willison's hands-on test is the only systematic third-party hands-on record, and it is one person on one task. The model is 7 days old, so the absence means "not yet", not "zero" — but until a score lands, every capability claim about Ember rests on one AA index number plus Meta's retelling. That is the first item on the 8-hour tracking loop's watchlist: first LiveBench / LMArena listings, and AA v5.x rescores.
The weak spots, collected: FrontierCode 2.0-AA loses to Haiku 5.5 (27.2% vs 34.9%); AA-Omniscience 21.1% is the lowest in its class; the Terminal-Bench 4.0 and OSWorld 2.1 leads are under 2 points; all six component benchmarks are Meta-reported, not an AA-published board; Meta published max-effort results only, leaving lower-effort quality officially unverified; no LiveBench / LMArena score; parameter count undisclosed. These don't invalidate 36 — but they set Ember's boundaries: business workflows and agent execution are worth trying, code-heavy work should check FrontierCode first, and lower effort levels should not be assumed equal to max.
Level with Claude Haiku 5.5 (max), below Muse Spark 1.3 (max) at 37; cost per task $0.14 vs Haiku $0.41 and Spark $2.39. AA calls it the "cheapest model scoring 36 or higher". It is the only index score AA published independently.
FrontierCode 2.0-AA: 27.2% vs 34.9%, a 7.7-point loss. AA-Omniscience 21.1% is also the lowest among models with similar overall scores. Both are weak spots in coding and general-capability dimensions.
No. They come from charts in Meta's launch post — Meta's retelling of AA evaluations; the AA Coding Agent Index of 46 is likewise Meta's figure. Only the Intelligence Index of 36 was independently published by AA on Nov 9.
No. As of Nov 16, LiveBench has no run and LMArena has no listing. The model is 7 days old; the absence means "not yet", not "zero".
Reference: Artificial Analysis — launch evaluation (Intelligence Index v5.0 36, $0.14 per task) · Meta research blog — six component benchmarks and the headline claim · alphasignal (AA-Omniscience 21.1%) · at-pulse (2nd of 33 comparable models) · Simon Willison hands-on test (Nov 10) (rewritten and organized by ModelBrief; facts per the AA evaluation and the Meta launch post)