ModelBrief · Kimi K3.1 Pro · Topic Page
Last updated Nov 20, 2026

Kimi K3.1 Pro Self-Hosting Guide: 1.56TB Weights, Cluster Bar

ModelBrief · Published 2026-11-16 · Fact-check 2026-11-16 · Reference: Hugging Face model card, Spheron deployment guide, vramglass

K3.1 Pro's weights are public, but "public" does not mean "anyone can run it": 96 shards at ~1.56TB, about 861GB of VRAM even at 2-bit. Consumer GPUs cannot run it; the floor is one 8×B300 node or four 8×H100 nodes. This article covers download, hardware, deployment frameworks and the license in one pass — with the conclusion first: most people should use the API, not self-host.

The weights first: 96 shards, ~1.56TB

Hugging Face repo moonshotai/Kimi-K3.1-Pro, public ungated download, live November 5, 2026 (Beijing time); the ModelScope mirror went live the same day (96/96 shards). 96 safetensor shards totalling 1,561,355,732,732 bytes, ~1.56TB; MXFP4 weights + MXFP8 activations, quantization-aware training (not post-hoc compression); an 8-bit compressed-tensors version is also provided. There is no GGUF as of November 4 — anyone hoping to run a quantized build on consumer hardware has to wait for a community GGUF, which does not exist yet. The download itself is a barrier: 1.56TB of weights means download time and storage costs must be budgeted first.

The hardware reality: ~861GB at 2-bit, ~1,509GB at 4-bit

vramglass estimates about 861GB of VRAM at 2-bit and 1,509GB at 4-bit. Spheron's deployment guide gives two configurations that hold it: a single 8×B300 node (2,304GB HBM3e), or four 8×H100 nodes. In other words, this is not a "few GPUs" problem but a "one node or a small cluster" problem; a 24GB consumer GPU is nowhere near enough. Note also that VRAM estimates are only the floor for holding the weights — real inference needs headroom for the KV cache and activations, and a 1M-context KV cache is itself a substantial VRAM cost, so size for full context with margin.

VRAM needs vs node capacity VRAM needed (estimate) Node capacity 2-bit weights ~861GB 4-bit weights ~1,509GB 8×B300 node 2,304GB VRAM estimates: vramglass; node sizing: Spheron guide; four 8×H100 nodes also work
VRAM needs (GB; vramglass estimates) vs single-node capacity; real inference also needs KV cache and activation headroom

Deployment frameworks: Transformers, vLLM, SGLang, Docker Model Runner

The official model card lists Transformers, vLLM, SGLang and Docker Model Runner. Reasoning effort is low / high / max, default max — when self-hosting, note that the default max effort produces the most reasoning tokens, so long-task output cost and latency should be evaluated at max. The card gives no ranked framework recommendation; Spheron's guide focuses on cluster deployment. One warning: the model-card scores are model-level results, not an evaluation of the downloadable weights — as the benchmarks article explains, the weights have never been independently tested, so scores from a self-hosted run may differ from the official blog's numbers. That is normal, not a deployment failure.

Hugging Face: moonshotai/Kimi-K3.1-Pro (96 shards, ~1.56TB; public download) ModelScope: mirror live (96/96 shards) Frameworks: Transformers / vLLM / SGLang / Docker Model Runner Reasoning: low / high / max (default max) Hardware: one 8×B300 node (2,304GB HBM3e) or four 8×H100 nodes License: Kimi K3.1-Pro License (custom, not MIT)

The license: read the terms before downloading

Kimi K3.1 Pro is not MIT-licensed — the most important thing to know before self-hosting. The license is the custom Kimi K3.1-Pro License: an MIT-style grant plus two commercial conditions — MaaS (model-as-a-service) revenue over $20M in any 12 months needs a separate agreement, and products with over 100M MAU or over $20M monthly revenue must display the model name. The predecessor Kimi K3 used Modified MIT, so K3.1 Pro changed the license — anyone upgrading a self-hosting setup from K3 should not carry over the old compliance judgement. Personal and small-scale use is unaffected by either condition; but before putting the model into a commercial product or serving it as an API, have legal read the full license text (attached to the model card).

Should you self-host? Answers for three kinds of users

Individuals and small teams: no. A 1.56TB download, cluster-scale hardware and KV-cache costs dwarf the $2/$10 API; Kimi.com and pay-as-you-go OpenRouter are the right choice. Teams that already own a GPU cluster: worth evaluating, but do two sums first — hardware depreciation and ops cost against the API bill, and the gap between self-hosted scores and the official numbers (the weights are untested independently). Teams with hard compliance or data-sovereignty requirements: self-hosting is the only option, though the Kimi API's optional Zero Data Retention is worth evaluating first; note the license's two commercial conditions, and that Highspeed has no separate weights repo (self-hosting can only run the standard tier).

FAQ

Can a consumer GPU run Kimi K3.1 Pro?

No. About 861GB of VRAM at 2-bit, 1,509GB at 4-bit; one 8×B300 node (2,304GB HBM3e) holds it, or four 8×H100 nodes. A 24GB consumer GPU is nowhere near enough.

Is there a GGUF build?

No official GGUF as of November 4. Whether a community build appears, and its quality, is a watch item on the 8-hour tracking loop.

What does the license restrict?

The custom Kimi K3.1-Pro License, not MIT: MaaS revenue over $20M in any 12 months needs a separate agreement; products with over 100M MAU or over $20M monthly revenue must display the model name. Personal and small-scale use is unaffected.

Will self-hosted scores match the official ones?

Not necessarily. The official blog's scores are model-level results, not a separate evaluation of the downloadable weights; the weights themselves have never been independently tested. A gap between self-hosted measurements and official numbers is normal.

ModelBrief's take: K3.1 Pro's self-hosting bar is among the highest on this site: 1.56TB weights, cluster-scale hardware and a custom license, three barriers at once. The point of the public weights is not that everyone can run them — it is that everyone can inspect them: researchers can check the weights, reproduce results and run their own evaluations. For most users the $2/$10 API is the right choice; self-hosting is for teams with a cluster and a compliance need.

Reference: Hugging Face — moonshotai/Kimi-K3.1-Pro model card (96 shards, MXFP4, license) · Spheron — deployment guide (B300/H100 sizing) · vramglass — VRAM estimates (rewritten and organized by ModelBrief; facts per the model card and the deployment guide)