pythia-160m-deduped
The Pythia Scaling Suite is a collection of models developed to facilitate interpretability research (see paper). It contains two sets of eight models of sizes 70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two models: one trained on the Pile, and one trained on the Pile after the dataset has been globally deduplicated. All 8 model sizes are trained on the exact same data, in the exact same...
Params
210 M
Context
2,048
Downloads 30d
492 K
Likes
4
Download history
daily snapshots · 53 days
▲ 120 K in the last 30 days (32.1%)
731 K433 K
Aug 23Sep 2Sep 12Sep 21
731 K162 K
Jul 31Aug 17Sep 4Sep 21
Can you run it?
Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.
| File | Quant | Size | Est. VRAM | Verdict on RTX 4090 · 24 GB |
|---|---|---|---|---|
| model.safetensors | f16 | 0.4 GB | 0.9 GB | ✅ Runs comfortably |
Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.
Run it
copy-paste, exact tags checked against the Hub$ curl -s https://aimodelscomparison.com/api/v1/models/pythia-160m-deduped
{
"hf_id": "EleutherAI/pythia-160m-deduped",
"params_b": 0.21,
"context_length": 2048,
"license": { "id": "apache-2.0", "commercial": "yes" },
"downloads_30d": 492009,
"vram_estimates": [
{ "quant": "f16", "gb": 0.9 }
],
"updated_at": "2026-07-31T01:00:38Z"
}
Specifications
- Architecture
- GPTNeoXForCausalLM
- Parameters
- 210 M
- Tensor type
- F16
- Context length
- 2,048
- Vocabulary
- 50,304
- Layers / heads
- 12 / 12
- Licence
- apache-2.0
- First seen on the Hub
- 2023-02-08
- Training datasets
- EleutherAI/the_pile_deduplicated
- Added to our catalog
- 2026-07-31
Compare with any text-generation model