pythia-6.9b
The Pythia Scaling Suite is a collection of models developed to facilitate interpretability research (see paper). It contains two sets of eight models of sizes 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two models: one trained on the Pile, and one trained on the Pile after the dataset has been globally deduplicated. All 8 model sizes are trained on the exact same data, in the exact same...
Params
7.0 B
Context
2,048
Downloads 30d
414 K
Likes
68
Download history
daily snapshots · 61 days
▲ 83 K in the last 30 days (16.6%)
496 K414 K
Oct 12Oct 22Nov 1Nov 10
760 K287 K
Sep 11Oct 1Oct 21Nov 10
Can you run it?
Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.
| File | Quant | Size | Est. VRAM | Verdict on RTX 4090 · 24 GB |
|---|---|---|---|---|
| model.safetensors | f16 | 13.8 GB | 16.8 GB | ✅ Runs comfortably |
Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.
Run it
copy-paste, exact tags checked against the Hub$ curl -s https://aimodelscomparison.com/api/v1/models/pythia-6-9b
{
"hf_id": "EleutherAI/pythia-6.9b",
"params_b": 6.99,
"context_length": 2048,
"license": { "id": "apache-2.0", "commercial": "yes" },
"downloads_30d": 413851,
"vram_estimates": [
{ "quant": "f16", "gb": 16.8 }
],
"updated_at": "2026-09-11T01:00:23Z"
}
Specifications
- Architecture
- GPTNeoXForCausalLM
- Parameters
- 7.0 B
- Tensor type
- F16
- Context length
- 2,048
- Vocabulary
- 50,432
- Layers / heads
- 32 / 32
- Licence
- apache-2.0
- First seen on the Hub
- 2023-02-14
- Training datasets
- EleutherAI/pile
- Added to our catalog
- 2026-09-11
Compare with any text-generation model