culturerevolt / text-generation updated 2 months ago

gemma-4-12b-heretic-abliterated-GGUF

This repository hosts a complete, professional-grade suite of iMatrix (Importance Matrix) GGUF quantizations of culturerevolt/gemma-4-12b-heretic-abliterated.

Params
Context
Downloads 30d
219 K
Likes
8
Commercial use: allowed apache-2.0 Not gated GGUF View on Hugging Face ↗

Download history

daily snapshots · 10 days
219 K177 K
Jul 28Jul 31Aug 3Aug 6

Can you run it?

Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.

FileQuantSizeEst. VRAMVerdict on RTX 4090 · 24 GB
gemma-4-12b-heretic-mmproj-f16.gguf GGUF 0.2 GB 0.7 GB ✅ Runs comfortably
gemma-4-12b-heretic-IQ3_XS.gguf IQ3_XS 5.3 GB 6.3 GB ✅ Runs comfortably
gemma-4-12b-heretic-IQ4_XS.gguf IQ4_XS 6.6 GB 7.8 GB ✅ Runs comfortably
gemma-4-12b-heretic-Q4_K_M.gguf Q4_K_M 7.4 GB 8.6 GB ✅ Runs comfortably
gemma-4-12b-heretic-Q5_K_M.gguf Q5_K_M 8.5 GB 9.9 GB ✅ Runs comfortably
gemma-4-12b-heretic-Q6_K.gguf Q6_K 9.8 GB 11.3 GB ✅ Runs comfortably
gemma-4-12b-heretic-abliterated-Q8_0.gguf Q8_0 12.7 GB 14.4 GB ✅ Runs comfortably
Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.

Run it

copy-paste, exact tags checked against the Hub
~ · ollama · Q4_K_M
$ ollama run gemma-4-12b-heretic-abliterated-gguf

# pin the quantization explicitly
$ ollama run gemma-4-12b-heretic-abliterated-gguf-q4_k_m
est. VRAM 8.6 GBon RTX 4090 · 24 GBJSON API →

Specifications

Licence
apache-2.0
First seen on the Hub
2026-06-05
Training datasets
undisclosed
Added to our catalog
2026-07-28
Sponsored · GPU cloud
Not enough VRAM?
Spin up a 24 GB L4 instance in 40 seconds. $0.44/hr.