facebook / feature-extraction updated 2 years ago

w2v-bert-2.0

Name: w2v-bert-2.0
Author: facebook

We are open-sourcing our Conformer-based W2v-BERT 2.0 speech encoder as described in Section 3.2.1 of the paper, which is at the core of our Seamless models.

Params

580 M

Context

—

Downloads 30d

1.4 M

Likes

225

Commercial use: allowed mit Not gated SAFETENSORS 96 languages View on Hugging Face ↗

Download history

daily snapshots · 10 days

1.6 M1.4 M

Jul 28Jul 31Aug 3Aug 6

Can you run it?

Estimated VRAM at 8K context unless noted. Pick your hardware to see the verdict per quantization.

File	Quant	Size	Est. VRAM	Verdict on RTX 4090 · 24 GB
model.safetensors	f32	2.3 GB	3.1 GB	✅ Runs comfortably

Estimate: file size × 1.1 + KV cache at 8K + 0.5 GB overhead. Not a benchmark — how we calculate this.

Specifications

Architecture: Wav2Vec2BertModel
Parameters: 580 M
Tensor type: F32
Layers / heads: 24 / 16
Licence: mit
First seen on the Hub: 2023-12-19
Training datasets: undisclosed
Added to our catalog: 2026-07-28

Compare with

bge-small-en-v1.5 72.4 M bge-large-en-v1.5 13.1 M Qwen3-Embedding-0.6B 9.8 M multilingual-e5-large 7.5 M jina-embeddings-v3 3.2 M all-MiniLM-L6-v2 2.9 M

Appears in

Best open coding models Commercial-use text models Models that fit in 12 GB