ModelBrief · Edge0-145B · Topic Page
Last updated Nov 20, 2026

Edge0-145B Architecture: Latent MoE, 512 Experts

ModelBrief · Published 2026-11-20 · Fact-check 2026-11-20 · Reference: Hugging Face model cards, Edge0 official blog, GitHub Edge0-AI/Edge0

Edge0-145B's selling point is not the parameter count itself but an architecture rewritten for one consumer computer: 145B total parameters with only 10.12B active per token, 48 layers with 512 experts each, a 51.2B-parameter n-gram embedding table, and a three-tier GPU / system-memory / SSD pipeline. This article works through the Hugging Face model card (read directly by this site on Nov 16, 2026) item by item — every number comes from the model card or the official blog, the technical report is still unpublished, and no third party has reproduced any of it yet.

One table: where the 145B parameters live

ComponentSizeNotes
Total parameters145B10.12B active per token — about 7% of parameters take part in a single step
Expert pool84.6B parametersLatent width ℓ=1792; each expert is a fixed 1.63 MB record
N-gram embedding table (PLE)51.2B parametersfp8, memory-mapped from SSD; about a third of all parameters
MTP speculative-decoding head1.26BDraft model with 512 experts, for speculative decoding
Rest (attention, vocabulary, etc.)About 7.9BResidual after the three items above

Read the composition first: expert pool 84.6B + n-gram table 51.2B + MTP head 1.26B ≈ 137B, leaving about 7.9B. More than a third of the parameters sit not in MoE experts but in an n-gram table mapped from SSD — the biggest structural difference between Edge0-145B and a conventional MoE.

Latent MoE: 48 layers, 512 experts per layer

Edge0-145B is a latent sparse MoE: 48 layers, 512 experts per layer, 10 routed experts + 1 shared expert active per token, SwiGLU width 640. Per the model card, the MoE expert layers were rebuilt — the base Qwen3.8-Flash-Next's routed experts were replaced by latent MoE in all 48 layers — and the model was post-trained afterwards on Edge0's own data. Attention is hybrid: full attention every 4 layers, with a sparse indexer on the rest.

Why "latent": experts do not sit in VRAM as full weight matrices. Each lives in the expert pool as a fixed 1.63 MB record, fetched on demand. Combined with the sparsity of 10 active experts per token, compute per step is held to 10.12B active parameters — the whole meaning of "a 145B model at 10B-scale compute". The cost is movement outside VRAM: expert records stream in from system memory or SSD, so speed depends on the pipeline, not on raw compute.

The three-tier pipeline: what lives on the GPU, in RAM, and on SSD

Edge0-145B does not run by loading the model into VRAM. It splits the work across three tiers: the GPU holds only what the current computation needs, system memory holds the expert library, and the SSD holds the n-gram vocabulary data. The model card's measured residency (RTX 5090 D 24GB) is: NVFP4 at 23.25 GiB VRAM + 38.27 GiB host RAM, Q3_K at 23.14 GiB VRAM + 31.28 GiB host RAM — plus an expert library of about 40GB in system memory and vocabulary data of about 48GB on SSD. The blog headline's "29GB peak memory" is GPU-side peak only: whole-machine demand is about 61.5 GiB for NVFP4 and about 54.4 GiB for Q3_K. See our install guide.

The point of the pipeline is a shifted threshold: it trades "needs 145B of VRAM" for "needs one 16GB+ VRAM GPU + 31–38 GiB of system RAM + about 40–48 GiB of disk". Good news for anyone with a big-memory desktop; not good news for a thin laptop with 16GB of unified memory — it fails the system-RAM test before it starts.

Native 4-bit training: not post-hoc quantization

According to Edge0, Edge0-145B was trained natively at 4 bits in the on-device architecture from the outset — not a post-hoc quantization of a cloud model. If true, that explains why it ships directly in two 4-bit builds, Q3_K and NVFP4: the weights were laid out at low precision during training, not trained in FP16 and squeezed afterwards. But note: this is the vendor's claim, the technical report is still unpublished, and training details (data volume, training steps, precision-loss comparisons) cannot be verified today. The two builds' measured speeds confirm the other side of "native" — Q3_K runs faster than NVFP4 (121.2 vs 98.3 tok/s agent decode), trading lower precision for higher throughput.

256K context with int8 KV

The context window is 256K tokens, with the KV cache in int8 and 32K resident. 256K is long for an on-device model — still short of the 1M cloud flagships, but enough for local agent work such as reading a code base or long-document Q&A. Int8 KV follows the same idea: squeeze VRAM further to buy a longer usable context. One caveat: the model card publishes no speed decay or accuracy figures at a full 256K load, so real long-context usability awaits independent testing.

Multimodal input and tool calling: present, with an extra helper

Input is text + image: the model folder carries a vision encoder, mmproj-BF16.gguf, but it needs the strata-vision helper and is not built from the Edge0 tree; output is text. Tool calling runs through the local OpenAI- / Anthropic-compatible server. In other words, vision and tool calling both exist, but the vision path needs an extra component — the launch-week documentation does not spell that out clearly, so check the strata-vision status before installing.

Measured speed: two builds, two score sheets

Build (RTX 5090 D 24GB, vendor-measured)Agent decode16K prefillVRAM / host RAM
NVFP4 (47.79 GiB)98.3 tok/s3,165.7 tok/s23.25 / 38.27 GiB
Q3_K (40.81 GiB)121.2 tok/s3,554.9 tok/s23.14 / 31.28 GiB

There is also a Metal measurement on Apple M2 Ultra: Q3_K at 57.0 tok/s agent decode — note this is a Metal speed measurement, not proof of macOS availability; the GitHub README's platform table lists no macOS support for the 145B/120B models at launch. See the install guide. The blog's headline figures (80+ tok/s decode, 1,100+ tok/s prefill, ~120W typical power) are a lower-bound summary across both builds — quote them with the build name and hardware.

ModelBrief's take: This architecture serves one explicit goal — fitting 145B into a single consumer computer. Latent MoE + a three-tier pipeline + native 4-bit training form a coherent story on paper: small active compute, small VRAM footprint, disk traded for memory. But there is a structural cost: a third of the parameters live in an SSD-mapped n-gram table, so speed depends on the fetch pipeline; and the "natively trained at 4 bits" claim has no technical report behind it yet. The architecture is credible; the verification is not here yet.