ESPnet Model Zoo

Pretrained ESPnet models, and the tools that keep them loadable

Models on Hugging Face · Registered models · ESPnet · ESPnet docs

---

espnet_model_zoo downloads a pretrained ESPnet model, unpacks it, and hands the resulting paths to the matching espnet2 inference class — so loading a model is one from_pretrained call rather than a checkpoint, a config and a token list to wire up yourself. The models live in the espnet organization on Hugging Face: 671 of them as of 2026-09-21, 624 carrying a pipeline_tag you can filter by. The design follows Asteroid's pretrained model function.

Important

Upgrade to 0.1.11 or later. Releases before it resolved paths through every string in a packed config, including the vocabulary, so a token that happened to name something inside the cache — OWSM's . and exp, for instance — was rewritten into an absolute path and came back in the transcript. 0.1.11 stops doing it and repairs a cache an older version already damaged, on load, with no re-download.

Install

pip install torch                # first, per https://pytorch.org/get-started/locally/
pip install espnet_model_zoo     # brings espnet with it

Quick start

A model name is a Hugging Face id (espnet/owsm_ctc_v4_1B), a tag from table.csv, a local .zip, or a Zenodo URL. Every task follows the same from_pretrained shape:

import soundfile
from espnet2.bin.asr_inference import Speech2Text

speech2text = Speech2Text.from_pretrained("model_name")
speech, rate = soundfile.read("speech.wav")   # at the model's training sample rate
text, *_ = speech2text(speech)[0]
print(text)

import soundfile
from espnet2.bin.tts_inference import Text2Speech

text2speech = Text2Speech.from_pretrained("model_name")
speech = text2speech("foobar")["wav"]
soundfile.write("out.wav", speech.numpy(), text2speech.fs, "PCM_16")

import soundfile
from espnet2.bin.enh_inference import SeparateSpeech

separate_speech = SeparateSpeech.from_pretrained("model_name")
speech, rate = soundfile.read("long_speech.wav")
waves = separate_speech(speech[None, ...], fs=rate)

Resample your audio to the rate the model was trained at; nothing does it for you.

Decoding and segmentation parameters

Decoding parameters are not stored in the model file, so pass them to from_pretrained:

speech2text = Speech2Text.from_pretrained(
    "model_name",
    maxlenratio=0.0,
    minlenratio=0.0,
    beam_size=20,
    ctc_weight=0.3,
    lm_weight=0.5,
    penalty=0.0,
    nbest=1,
)

SeparateSpeech handles both short and long audio. Segment-wise processing is off by default; segment_size and hop_size turn it on, and normalize_segment_scale and show_progressbar tune it:

separate_speech = SeparateSpeech.from_pretrained(
    "model_name",
    segment_size=2.4,
    hop_size=0.8,
    normalize_segment_scale=False,
    show_progressbar=True,
    ref_channel=None,
    normalize_output_wav=True,
)

The API before ESPnet 0.10.1

ASR

import soundfile
from espnet_model_zoo.downloader import ModelDownloader
from espnet2.bin.asr_inference import Speech2Text
d = ModelDownloader()
speech2text = Speech2Text(
    **d.download_and_unpack("model_name"),
    # Decoding parameters are not included in the model file
    maxlenratio=0.0,
    minlenratio=0.0,
    beam_size=20,
    ctc_weight=0.3,
    lm_weight=0.5,
    penalty=0.0,
    nbest=1
)

TTS

import soundfile
from espnet_model_zoo.downloader import ModelDownloader
from espnet2.bin.tts_inference import Text2Speech
d = ModelDownloader()
text2speech = Text2Speech(**d.download_and_unpack("model_name"))

Speech separation

import soundfile
from espnet_model_zoo.downloader import ModelDownloader
from espnet2.bin.enh_inference import SeparateSpeech
d = ModelDownloader()
separate_speech = SeparateSpeech(
    **d.download_and_unpack("model_name"),
    # for segment-wise process on long speech
    segment_size=2.4,
    hop_size=0.8,
    normalize_segment_scale=False,
    show_progressbar=True,
    ref_channel=None,
    normalize_output_wav=True,
)

Find a model

Filter the Hugging Face organization by task, or query table.csv locally:

from espnet_model_zoo.downloader import ModelDownloader

d = ModelDownloader()
d.query("name")                    # every registered name
d.query("name", task="asr")        # narrowed by any column of table.csv

espnet_model_zoo_query                              # all names
espnet_model_zoo_query task=asr corpus=wsj          # narrowed
espnet_model_zoo_query --key url task=asr corpus=wsj

Download and cache

from espnet_model_zoo.downloader import ModelDownloader

d = ModelDownloader()                    # ~/.cache/espnet_model_zoo; Hugging Face
                                         # models go to the huggingface_hub cache
d = ModelDownloader("~/.cache/espnet")   # or choose the directory

download_and_unpack returns the paths an inference class needs, and skips the work if the model is already there:

>>> d.download_and_unpack("kamo-naoyuki/mini_an4_asr_train_raw_bpe_valid.acc.best")
{"asr_train_config": <config path>, "asr_model_file": <model path>, ...}

It takes the same four kinds of name as from_pretrained, plus a query:

d.download_and_unpack("kamo-naoyuki/mini_an4_...@<revision>")  # a Hub revision
d.download_and_unpack("https://zenodo.org/record/...")         # a URL
d.download_and_unpack("./some/where/model.zip")                # a local file
d.download_and_unpack(task="asr", corpus="wsj")                # a query: last match
d.download_and_unpack(task="asr", corpus="wsj", version=-2)    # the one before it

A local file is unpacked into the cache too, and is identified by its path — move it and unpack again and it is treated as a different model, expanded a second time.

If a model was uploaded to the Hub by hand rather than by a recipe, it has no meta.yaml saying which file is the config and which is the checkpoint. download_and_unpack then fails with a RuntimeError listing the repository's files, and you pass train_config and model_file yourself. Tell us which model it was — repairing those in place is a maintainer job, described in MAINTAINING.md.

espnet_model_zoo_download <model_name>                # prints the downloaded file
espnet_model_zoo_download --unpack true <model_name>  # prints the unpacked files

Use a model in an ESPnet recipe

# e.g. ASR WSJ task
git clone https://github.com/espnet/espnet
pip install -e .
cd egs2/wsj/asr1
./run.sh --skip_data_prep false --skip_train true --download_model kamo-naoyuki/wsj

Publish your model

Upload from the recipe that trained it, then register it here.

1. Create a Hugging Face account and a new model repository. Name it after the recipe and the model, e.g. aidatatang_200zh_conformer.

2. From the recipe, push the trained model:

   ./run.sh --stage 15 --skip_upload_hf false --hf_repo <user>/aidatatang_200zh_conformer

   The stage number is the upload stage of that task's pipeline — 15 for asr1, other tasks differ, so check ./run.sh --help.

3. Open a pull request adding a row to table.csv, so the model is covered by CI. A Hugging Face id identifies the model by itself, so the url column is just https://huggingface.co/:

   aidatatang_200zh,asr,sw005320/aidatatang_200zh_conformer,https://huggingface.co/,16000,zh,,,,,true
   

4. An administrator increments the third version number in setup.py and releases.

Screenshots of the Hub steps

Creating an account:

Creating the model repository:

A successful upload:

Zenodo (obsolete)

1. Upload your model to Zenodo

   You need to signup to Zenodo and create an access token to upload models. You can upload your own model by using espnet_model_zoo_upload command freely, but we normally upload a model using recipes.

2. Create a Pull Request to modify table.csv

   You need to append your record at the last line.

3. (Administrator does) Increment the third version number of setup.py, e.g. 0.0.3 -> 0.0.4

4. (Administrator does) Release new version

export ACCESS_TOKEN=<access_token>
espnet_model_zoo_upload \
    --file <packed_model> \
    --title <title> \
    --description <description> \
    --creator_name <your-git-account>

---

Maintaining the Hugging Face organization: MAINTAINING.md · Released under the Apache 2.0 License.