ModelBrief · Nex-N2.6 · Topic Page
Last updated Nov 20, 2026

Nex-N2.6 Self-Hosting Guide: 328GB FP8 Weights, No Consumer GPU

ModelBrief · Published 2026-11-16 · Fact-check 2026-11-16 · Reference: Hugging Face model card

Nex-N2.6's weights are Apache 2.0 open — anyone can download them. But "downloadable" is not "runnable": FP8 weights run ~328GB, ~202.6GB even at 4-bit, far beyond any consumer GPU. This article settles three things: what the weights are, how to deploy them, and what the hardware reality is; plus how to choose between the MOSS and OpenRouter alternatives when self-hosting doesn't fit.

What the weights are: 129 shards, ~328GB

Hugging Face repo nex-agi/Nex-N2.6, Apache 2.0 license: FP8 weights in 129 safetensor shards, ~328.34GB — 314.4B parameters in FP8, 6.92B in BF16, 3.6M in FP32 (shard detail from nlp.com.ai). The weights became publicly readable only around Nov 12 (the HF repo was still unreadable on the Nov 10 release day). Check disk space before downloading: 328GB is the weights alone, before extraction, cache and KV cache headroom. And remember the card's statement: the benchmark scores are model-level results, not a separate evaluation of the FP8 export — locally run scores have no independent verification.

How to deploy: SGLang, FP8 config

The recommended runtime is SGLang (FP8 config), compatible with the GLM-5.3-Flash runtime — N2.6 is a post-trained derivative of GLM-5.3-Flash, so deployment follows the base. Reasoning is mandatory: the HF chat template offers reasoning_effort low/high/max, default max; the OpenRouter endpoint offers medium/high, default high — don't mix the two tier systems. Context is 1,048,576 tokens per the released config (the OpenRouter endpoint gives only 262,144); self-hosting reaches 1M per the config, but KV cache usage rises sharply with long context — plan capacity against your actual context needs.

The hardware reality: a 512GB Mac Studio or multi-GPU server is the floor

The conclusion first: a 24GB consumer GPU is nowhere near enough. FP8 weights are ~328GB (~321GB by lininn.cn's estimate), ~202.6GB at 4-bit — far beyond any single consumer card even quantized. The realistic floor is a 512GB-unified-memory Mac Studio or a multi-GPU server (madebyagents' hardware-fit analysis and lininn.cn's reality check agree). Leave room for KV cache too: mandatory reasoning plus long context means both reasoning tokens and cache usage run large — budget VRAM/memory for weights plus KV cache together. Without hardware at that scale, self-hosting isn't the money-saving option — the API is cheaper.

Alternatives: free MOSS and pay-as-you-go OpenRouter

When self-hosting doesn't fit, two API channels are available: the MOSS API, free for a limited time (endpoint api.mosi.cn/v1, model nex-n2.6, 0 credits per request, no end date, no SLA), good for trying out and small-scale tests; and OpenRouter pay-as-you-go ($0.12/$0.40 per 1M tokens, cache reads $0.024), good for production — but with Nex-AGI itself as the sole provider and no fallback endpoint. If you need high availability without self-hosting, there is currently no perfect answer: the free window promises no SLA, and the paid channel has one provider. See our pricing guide.

FAQ

Can a 24GB GPU run N2.6?

No. FP8 weights are ~328GB, ~202.6GB at 4-bit — far beyond 24GB. The realistic floor is a 512GB-unified-memory Mac Studio or a multi-GPU server.

Which framework for self-hosting?

SGLang (FP8 config), compatible with the GLM-5.3-Flash runtime. Reasoning is mandatory; the HF template's reasoning_effort defaults to max.

Can self-hosting use 1M context?

Per the released config, yes (1,048,576 tokens) — but KV cache usage rises sharply with long context, so plan capacity against actual needs; and 1M-context behaviour has no independent verification.

ModelBrief's take: N2.6's open weights are real, and so is the self-hosting bar: 328GB of weights, mandatory reasoning, long-context KV cache — stacked together, self-hosting is out of reach for most individuals. For most people, the realistic route is the free MOSS window for trials and pay-as-you-go OpenRouter for production; self-hosting only fits teams that already have large-memory hardware.

Reference: Hugging Face — nex-agi/Nex-N2.6 model card (FP8 weights and config) · lininn.cn — hardware reality check (~202.6GB at 4-bit) (rewritten and organized by ModelBrief; facts per the model card and the original analysis)