pythia-160m
The Pythia Scaling Suite is a collection of models developed to facilitate interpretability research (see paper). It contains two sets of eight models of sizes 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two models: one trained on the Pile, and one trained on the Pile after the dataset has been globally deduplicated. All 8 model sizes are trained on the exact same data, in the exact same...
Params
210 M
Context
2,048
Downloads 30d
3.5 M
Likes
66
Download history
daily snapshots · 106 days
▲ 35 K in the last 30 days (1.0%)
3.5 M3.3 M
Oct 12Oct 22Nov 1Nov 10
3.5 M2.3 M
Aug 13Sep 12Oct 12Nov 10
3.5 M2.3 M
Jul 28Sep 1Oct 6Nov 10
Can you run it?
Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.
| File | Quant | Size | Est. VRAM | Verdict on RTX 4090 · 24 GB |
|---|---|---|---|---|
| model.safetensors | f16 | 0.4 GB | 0.9 GB | ✅ Runs comfortably |
Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.
Run it
copy-paste, exact tags checked against the Hub$ curl -s https://aimodelscomparison.com/api/v1/models/pythia-160m
{
"hf_id": "EleutherAI/pythia-160m",
"params_b": 0.21,
"context_length": 2048,
"license": { "id": "apache-2.0", "commercial": "yes" },
"downloads_30d": 3495753,
"vram_estimates": [
{ "quant": "f16", "gb": 0.9 }
],
"updated_at": "2026-07-28T17:07:55Z"
}
Specifications
- Architecture
- GPTNeoXForCausalLM
- Parameters
- 210 M
- Tensor type
- F16
- Context length
- 2,048
- Vocabulary
- 50,304
- Layers / heads
- 12 / 12
- Licence
- apache-2.0
- First seen on the Hub
- 2023-02-08
- Training datasets
- EleutherAI/pile
- Added to our catalog
- 2026-07-28
Compare with any text-generation model
Popular comparisons
Appears in