pythia-160m
The Pythia Scaling Suite is a collection of models developed to facilitate interpretability research (see paper). It contains two sets of eight models of sizes 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two models: one trained on the Pile, and one trained on the Pile after the dataset has been globally deduplicated. All 8 model sizes are trained on the exact same data, in the exact same...
Params
210 M
Context
2,048
Downloads 30d
3.5 M
Likes
45
Download history
daily snapshots · 55 days
▲ 15 K in the last 30 days (0.4%)
3.5 M3.4 M
Aug 22Sep 1Sep 11Sep 20
3.5 M2.3 M
Jul 28Aug 15Sep 2Sep 20
Can you run it?
Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.
| File | Quant | Size | Est. VRAM | Verdict on RTX 4090 · 24 GB |
|---|---|---|---|---|
| model.safetensors | f16 | 0.4 GB | 0.9 GB | ✅ Runs comfortably |
Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.
Run it
copy-paste, exact tags checked against the Hub$ curl -s https://aimodelscomparison.com/api/v1/models/pythia-160m
{
"hf_id": "EleutherAI/pythia-160m",
"params_b": 0.21,
"context_length": 2048,
"license": { "id": "apache-2.0", "commercial": "yes" },
"downloads_30d": 3470216,
"vram_estimates": [
{ "quant": "f16", "gb": 0.9 }
],
"updated_at": "2026-07-28T17:07:55Z"
}
Specifications
- Architecture
- GPTNeoXForCausalLM
- Parameters
- 210 M
- Tensor type
- F16
- Context length
- 2,048
- Vocabulary
- 50,304
- Layers / heads
- 12 / 12
- Licence
- apache-2.0
- First seen on the Hub
- 2023-02-08
- Training datasets
- EleutherAI/pile
- Added to our catalog
- 2026-07-28
Compare with any text-generation model
Popular comparisons
Appears in