culturerevolt / text-generation updated 3 months ago

gemma-4-12b-heretic-abliterated-GGUF

This repository hosts a complete, professional-grade suite of iMatrix (Importance Matrix) GGUF quantizations of culturerevolt/gemma-4-12b-heretic-abliterated.

Params
Context
Downloads 30d
196 K
Likes
29
Commercial use: allowed apache-2.0 Not gated GGUF View on Hugging Face ↗

Download history

daily snapshots · 57 days
▲ 80 K in the last 30 days (28.9%)
304 K177 K
Jul 28Aug 16Sep 4Sep 22

Can you run it?

Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.

FileQuantSizeEst. VRAMVerdict on RTX 4090 · 24 GB
gemma-4-12b-heretic-mmproj-f16.gguf GGUF 0.2 GB 0.7 GB ✅ Runs comfortably
gemma-4-12b-heretic-IQ3_XS.gguf IQ3_XS 5.3 GB 6.3 GB ✅ Runs comfortably
gemma-4-12b-heretic-IQ4_XS.gguf IQ4_XS 6.6 GB 7.8 GB ✅ Runs comfortably
gemma-4-12b-heretic-Q4_K_M.gguf Q4_K_M 7.4 GB 8.6 GB ✅ Runs comfortably
gemma-4-12b-heretic-Q5_K_M.gguf Q5_K_M 8.5 GB 9.9 GB ✅ Runs comfortably
gemma-4-12b-heretic-Q6_K.gguf Q6_K 9.8 GB 11.3 GB ✅ Runs comfortably
gemma-4-12b-heretic-abliterated-Q8_0.gguf Q8_0 12.7 GB 14.4 GB ✅ Runs comfortably
Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.

Run it

copy-paste, exact tags checked against the Hub
~ · ollama · Q4_K_M
$ ollama run gemma-4-12b-heretic-abliterated-gguf

# pin the quantization explicitly
$ ollama run gemma-4-12b-heretic-abliterated-gguf-q4_k_m
est. VRAM 8.6 GBon RTX 4090 · 24 GBJSON API →

Specifications

Licence
apache-2.0
First seen on the Hub
2026-06-05
Training datasets
undisclosed
Added to our catalog
2026-07-28
Compare with any text-generation model