Skip to content

README.md

Latest commit

 

History

History
65 lines (45 loc) · 2.39 KB

README.md

File metadata and controls

65 lines (45 loc) · 2.39 KB
Raw

Notes

Dataset

For more information about the dataset, please refer to these papers:

  • Renkens, Vincent, et al. "Acquisition of ordinal words using weakly supervised NMF." 2014 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2014.

  • Renkens, Vincent, and Hugo Van hamme. "Capsule networks for low resource spoken language understanding." arXiv preprint arXiv:1805.02922 (2018).

We followed this paper to randomly split the entire dataset into train/dev/test sets. Please modify the data preparation script to make the procedure deterministic. Otherwise the results cannot be exactly reproduced. Since this dataset is small, it might be better to run multiple experiments and calculate the average accuracy.

  • Tian, Yusheng, and Philip John Gorinski. "Improving end-to-end speech-to-intent classification with Reptile." arXiv preprint arXiv:2008.01994 (2020).

Experiments

Each command is considered as a single token. Thus, each reference sentence contains only one token.

feats_type: raw

Environments

  • date: Sat Nov 20 16:16:36 EST 2021
  • python version: 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0]
  • espnet version: espnet 0.10.5a1
  • pytorch version: pytorch 1.9.0
  • Git hash: 09ddefe8fd5b6394338b0c653c3f6ec50063a843
    • Commit date: Sat Nov 20 15:56:30 2021 -0500

asr_conformer_mono16k_warmup800_lr2e-4_accum2

Model: https://zenodo.org/record/5716386#.YcZQgxOZMVU

WER

dataset Snt Wrd Corr Sub Del Ins Err S.Err
infer/dev 1584 1584 97.9 2.1 0.0 0.0 2.1 2.1
infer/test 3631 3631 97.6 2.4 0.0 0.0 2.4 2.4

feats_type: fbank_pitch

Environments

  • date: Fri Oct 1 11:24:37 UTC 2021
  • python version: 3.8.12 | packaged by conda-forge | (default, Sep 16 2021, 02:08:29) [GCC 9.4.0]
  • espnet version: espnet 0.10.3a3
  • pytorch version: pytorch 1.9.0
  • Git hash: 3d17c072348a1a9a4a3f179ad642c0d9f07f4406
    • Commit date: Fri Oct 1 05:46:21 2021 +0000

asr_conformer_mono16k_warmup800_lr2e-4_accum2

Model: https://zenodo.org/record/5637566#.YcZQOxOZMVU

Classification Accuracy

dataset total correct accuracy
dev 1584 1529 0.965
test 3631 3528 0.972

WER

dataset Snt Wrd Corr Sub Del Ins Err S.Err
infer/dev 1584 1584 96.5 3.5 0.0 0.0 3.5 3.5
infer/test 3631 3631 97.2 2.8 0.0 0.0 2.8 2.8