janhq / text-generation updated 4 months ago

Jan-v3.5-4B-gguf

Name: Jan-v3.5-4B-gguf
Author: janhq

Jan-v3.5-4B is a fine-tuned variant of Jan-v3-4B-base-instruct, specialized on math reasoning and identity datasets. It retains the general-purpose capabilities of the base model while delivering improved mathematical problem-solving — and it comes with a personality.

Params

—

Context

—

Downloads 30d

316 K

Likes

Commercial use: allowed apache-2.0 Not gated GGUF 1 languages View on Hugging Face ↗

Download history

daily snapshots · 10 days

318 K310 K

Jul 28Jul 31Aug 3Aug 6

Can you run it?

Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.

File	Quant	Size	Est. VRAM	Verdict on RTX 4090 · 24 GB
Jan-v3.5-4B-Q3_K_S.gguf	Q3_K_S	2.1 GB	2.8 GB	✅ Runs comfortably
Jan-v3.5-4B-Q3_K_M.gguf	Q3_K_M	2.2 GB	3.0 GB	✅ Runs comfortably
Jan-v3.5-4B-Q3_K_L.gguf	Q3_K_L	2.4 GB	3.1 GB	✅ Runs comfortably
Jan-v3.5-4B-Q4_0.gguf	Q4_0	2.6 GB	3.3 GB	✅ Runs comfortably
Jan-v3.5-4B-Q4_K_S.gguf	Q4_K_S	2.6 GB	3.4 GB	✅ Runs comfortably
Jan-v3.5-4B-Q4_K_M.gguf	Q4_K_M	2.7 GB	3.5 GB	✅ Runs comfortably
Jan-v3.5-4B-Q4_1.gguf	Q4_1	2.8 GB	3.6 GB	✅ Runs comfortably
Jan-v3.5-4B-Q4_K_XL.gguf	Q4_K_XL	3.0 GB	3.8 GB	✅ Runs comfortably

Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.

Run it

copy-paste, exact tags checked against the Hub

~ · ollama · Q4_0

$ ollama run jan-v3-5-4b-gguf

# pin the quantization explicitly
$ ollama run jan-v3-5-4b-gguf-q4_0

est. VRAM 3.3 GBon RTX 4090 · 24 GBJSON API →

Specifications

Licence: apache-2.0
First seen on the Hub: 2026-03-23
Training datasets: undisclosed
Added to our catalog: 2026-07-28

Appears in

Best open coding models Commercial-use text models Models that fit in 12 GB