Add ECAPA2 to VoxCeleb by othman-istaiteh · Pull Request #3039 · speechbrain/speechbrain

othman-istaiteh · 2026-03-04T10:26:36Z

What does this PR do?

This PR implements the ECAPA2 model architecture and its corresponding training recipe for VoxCeleb.

Key Additions:

speechbrain/lobes/models/ECAPA2.py: Implementation of the ECAPA2 architecture and SubCenterClassifier.
speechbrain/nnet/losses.py: Added JeffreysLoss for embedding regularization.
VoxCeleb Recipe (recipes/VoxCeleb/SpeakerRec/):
- Added train_ecapa2.yaml and verification_ecapa2.yaml.
- Updated train_speaker_embeddings.py and speaker_verification_cosine.py to support the new model and pipeline requirements.
- Handled backward compatibility natively; existing models (X-Vector, ResNet, ECAPA-TDNN) run without modification.

Testing & Validation:

Added ECAPA2 testing vectors to tests/recipes/VoxCeleb.csv.
Ran pytest tests to ensure existing functionality remains intact.
Passed all doctests.
Ran pre-commit run -a to verify strict code formatting and linting.

Performance:

Trained on VoxCeleb 1 + VoxCeleb 2:

VoxCeleb1-O: 0.60% EER (with s-norm) / 0.70% EER (without s-norm)

Trained on VoxCeleb 2 only (tested without s-norm):

VoxCeleb1-O: 0.79% EER
VoxCeleb1-E: 1.00% EER
VoxCeleb1-H: 1.76% EER

Fixes N/A

Breaking changes: None. Backward compatibility is maintained for existing VoxCeleb scripts.

Before submitting

Did you read the contributor guideline?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Does your code adhere to project-specific code style and conventions?

PR review

Reviewer checklist

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified
Confirm that the changes adhere to compatibility requirements (e.g., Python version, platform)
Review the self-review checklist to ensure the code is ready for review

…tibility

TParcollet

Hey! Thank you very much for this recipe! I won't be able to try it because we do not have the voxceleb data. Before finding someone to try it, could you please address the comments?

TParcollet · 2026-05-27T11:55:17Z

+    num_workers: !ref <num_workers>
+
+# Functions
+use_tacotron2_mel_spec: True


Why using this? Is there a reason for not using standard Mels?

No, I just followed the same used in this recipe https://github.com/speechbrain/speechbrain/blob/develop/recipes/VoxCeleb/SpeakerRec/hparams/train_ecapa_tdnn_mel_spec.yaml

But I can change it to standard mels

Oh, this is new, interesting, then ok. Could you try with the standard ones just to see the end result maybe?

TParcollet · 2026-05-27T11:55:52Z

@@ -0,0 +1,97 @@
+# ################################
+# Model: Speaker Verification Baseline for ECAPA2
+# Acknowledgment: The source code is derived from the Kiwano toolkit.


Add author name for tracking.

TParcollet · 2026-05-27T11:56:20Z

 |-----------------|------------|------| -----|
 | Xvector + PLDA  | VoxCeleb 1,2 | 3.23% | https://www.dropbox.com/sh/ab1ma1lnmskedo8/AADsmgOLPdEjSF6wV3KyhNG1a?dl=0 |
 | ECAPA-TDNN      | VoxCeleb 1,2 | 0.80% | https://www.dropbox.com/sh/ab1ma1lnmskedo8/AADsmgOLPdEjSF6wV3KyhNG1a?dl=0 |
+| ECAPA2          | VoxCeleb 1,2 | 0.60% | https://drive.google.com/drive/folders/1cpU5qpCVM30Ip8I85EPM33lsUYPa6S7q?usp=sharing |


Please @Adel-Moumen can we upload this to dropbox?

TParcollet · 2026-05-27T11:56:57Z

    with torch.no_grad():
-        feats = params["compute_features"](wavs)
+        if (
+            "use_tacotron2_mel_spec" in params


Yes, this is a bit confusing. See above question, i'd prefer if we could use standard Mel. Is there a real difference?

TParcollet · 2026-05-27T11:58:55Z

+
+
+class SubCenterClassifier(nn.Module):
+    """Sub-Center ArcFace Classifier.


Docstring isn't explicit enough. I don't know what this is.

TParcollet · 2026-05-27T11:59:50Z

+
+
+class ECAPA2Res2NetConv1d(nn.Module):
+    """Res2Net convolutional block for 1D features."""


TParcollet · 2026-05-27T11:59:59Z

+
+
+class ECAPA2TDNNBlock(nn.Module):
+    """TDNN block for ECAPA2."""


TParcollet · 2026-05-27T12:00:05Z

+
+
+class ECAPA2DenseBlock(nn.Module):
+    """Dense convolutional block for ECAPA2."""


TParcollet · 2026-05-27T12:00:11Z

+
+
+class ECAPA2AttentiveStatPoolingBlock(nn.Module):
+    """Attentive Statistics Pooling for ECAPA2."""


TParcollet · 2026-05-27T12:00:45Z

+
+
+class JeffreysLoss(nn.Module):
+    """Computes the Jeffreys Loss, a combination of Cross Entropy, Label Smoothing,


Can we get a unit test for this new loss please?

othman-istaiteh added 6 commits March 4, 2026 10:48

Add ECAPA2 model architecture and SubCenterClassifier

1a37231

Add JeffreysLoss for speaker embedding regularization

0b0a488

Update VoxCeleb scripts to support ECAPA2 and maintain backward compa…

2645d52

…tibility

Add ECAPA2 hyperparameter configs and register in CI testing

2b910c1

Update VoxCeleb README with ECAPA2 performance results and instructions

a70c70d

Add attribution to the Kiwano toolkit

b43ab2a

othman-istaiteh force-pushed the add-ecapa2 branch from be08a77 to b43ab2a Compare March 4, 2026 15:35

othman-istaiteh and others added 2 commits March 4, 2026 16:39

Merge branch 'develop' into add-ecapa2

a5df5d3

Acknowledge Kiwano toolkit for ECAPA2 implementation

b1c9b74

othman-istaiteh marked this pull request as ready for review March 4, 2026 18:39

TParcollet requested changes May 27, 2026

View reviewed changes



		class SubCenterClassifier(nn.Module):
		"""Sub-Center ArcFace Classifier.



		class ECAPA2Res2NetConv1d(nn.Module):
		"""Res2Net convolutional block for 1D features."""



		class ECAPA2TDNNBlock(nn.Module):
		"""TDNN block for ECAPA2."""



		class ECAPA2DenseBlock(nn.Module):
		"""Dense convolutional block for ECAPA2."""



		class ECAPA2AttentiveStatPoolingBlock(nn.Module):
		"""Attentive Statistics Pooling for ECAPA2."""



		class JeffreysLoss(nn.Module):
		"""Computes the Jeffreys Loss, a combination of Cross Entropy, Label Smoothing,

Conversation

othman-istaiteh commented Mar 4, 2026

What does this PR do?

PR review

Uh oh!

TParcollet left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TParcollet May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

TParcollet May 27, 2026 •

edited

Loading