vcruz305 / text-generation updated 3 weeks ago

Hy3-GGUF

Name: Hy3-GGUF
Author: vcruz305

imatrix GGUF quantizations of tencent/Hy3 — 295B total / 21B active MoE (192 experts, top-8), 80 layers + 1 MTP/NextN layer (3.8B), 256K context.

Params

—

Context

—

Downloads 30d

891 K

Likes

Commercial use: allowed apache-2.0 Not gated GGUF View on Hugging Face ↗

Download history

daily snapshots · 10 days

891 K888 K

Jul 28Jul 31Aug 3Aug 6

Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.

File	Quant	Size	Est. VRAM	Verdict on RTX 4090 · 24 GB
Hy3-IQ4_XS-00001-of-00004.gguf	IQ4_XS	47.7 GB	52.9 GB	❌ Won’t fit
Hy3-Q4_K_M-00002-of-00004.gguf	Q4_K_M	47.6 GB	52.9 GB	❌ Won’t fit
Hy3-IQ1_M-00001-of-00002.gguf	IQ1_M	47.7 GB	53.0 GB	❌ Won’t fit
Hy3-IQ3_XXS-00001-of-00003.gguf	IQ3_XXS	47.8 GB	53.0 GB	❌ Won’t fit
Hy3-Q3_K_M-00002-of-00003.gguf	Q3_K_M	47.7 GB	53.0 GB	❌ Won’t fit
Hy3-Q5_K_M-00003-of-00005.gguf	Q5_K_M	47.9 GB	53.2 GB	❌ Won’t fit
Hy3-IQ2_M-00002-of-00003.gguf	IQ2_M	48.0 GB	53.3 GB	❌ Won’t fit
Hy3-Q2_K-00001-of-00003.gguf	Q2_K	48.0 GB	53.3 GB	❌ Won’t fit

Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.

copy-paste, exact tags checked against the Hub

~ · ollama · Q4_K_M

$ ollama run hy3-gguf

# pin the quantization explicitly
$ ollama run hy3-gguf-q4_k_m

est. VRAM 52.9 GBon RTX 4090 · 24 GBJSON API →

Appears in

Best open coding models Commercial-use text models Models that fit in 12 GB