unsloth / text-generation updated 1 year ago

Qwen3-235B-A22B-GGUF

Name: Qwen3-235B-A22B-GGUF
Author: unsloth

Unsloth Dynamic 2.0 achieves superior accuracy & outperforms other leading quants.

Params

—

Context

40,960

Downloads 30d

184 K

Likes

Commercial use: allowed apache-2.0 Not gated GGUF View on Hugging Face ↗

Download history

daily snapshots · 10 days

184 K173 K

Jul 28Jul 31Aug 3Aug 6

Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.

File	Quant	Size	Est. VRAM	Verdict on RTX 4090 · 24 GB
Qwen3-235B-A22B-BF16-00003-of-00010.gguf	GGUF	49.8 GB	55.2 GB	❌ Won’t fit
Qwen3-235B-A22B-Q2_K_L-00001-of-00002.gguf	Q2_K_L	49.7 GB	55.2 GB	❌ Won’t fit
Qwen3-235B-A22B-Q3_K_M-00001-of-00003.gguf	Q3_K_M	49.8 GB	55.3 GB	❌ Won’t fit
Qwen3-235B-A22B-Q4_1-00002-of-00003.gguf	Q4	49.8 GB	55.3 GB	❌ Won’t fit
Qwen3-235B-A22B-Q2_K-00001-of-00002.gguf	Q2_K	49.9 GB	55.4 GB	❌ Won’t fit
Qwen3-235B-A22B-Q4_K_M-00001-of-00003.gguf	Q4_K_M	49.9 GB	55.4 GB	❌ Won’t fit
Qwen3-235B-A22B-IQ4_XS-00001-of-00003.gguf	IQ4_XS	50.0 GB	55.5 GB	❌ Won’t fit
Qwen3-235B-A22B-Q3_K_S-00002-of-00003.gguf	Q3_K_S	50.0 GB	55.5 GB	❌ Won’t fit

Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.

copy-paste, exact tags checked against the Hub

~ · ollama · Q4

$ ollama run qwen3-235b-a22b-gguf

# pin the quantization explicitly
$ ollama run qwen3-235b-a22b-gguf-q4

est. VRAM 55.3 GBon RTX 4090 · 24 GBJSON API →

Base model and the most-downloaded derivatives in the catalog.

Appears in

Best open coding models Commercial-use text models Models that fit in 12 GB