DA-MLDG

On efficacy of Meta-Learning for Domain Generalization in Speech Emotion Recognition

Raeshak King Gandhi · Vasileios Tsouvalas · Nirvana Meratnia

Abstract

Domain-shift Aware Meta-Learning for Domain Generalization (DA-MLDG) studies how speech emotion models transfer to unseen corpora, speakers and languages. Each step divides the source domains into meta-train and meta-test groups, adapts a ResNet-18 on the first, then weights the second group's loss by their energy-score gap. The final target corpus remains outside training. Evaluation covers ten speech corpora, seven transfer scenarios and a separate PACS image benchmark, comparing Deep-All, MLDG and RSC. Stronger pretrained audio representations improve recognition, but cross-corpus generalization remains uneven.

Setup

Create and activate the damldg environment:

git clone https://github.com/Nousphera/DA-MLDG.git
cd DA-MLDG
python3.12 -m venv --prompt damldg "$HOME/.venvs/damldg"
source "$HOME/.venvs/damldg/bin/activate"

From the activated damldg environment, install the requirements:

python -m pip install -r requirements.txt

Data and Weights

Download and prepare the corpora for a transfer scenario:

# TESS + EmoDB → ShEMO
python scripts/prepare_data.py --scenario scn7

# One corpus, or the complete collection
python scripts/prepare_data.py --dataset ravdess
python scripts/prepare_data.py

For access-controlled corpora, download from the provider and retain these layouts:

Dataset	Provider	Location Under raw/
SAVEE	Surrey	savee/DC/a01.wav (DC, JE, JK, KL)
IEMOCAP	USC	iemocap/Session1/… through Session5
EMOVO	FUB	emovo/f1/rab-f1-b1.wav (f1–f3, m1–m3)
AESDD	AUTH	aesdd/AESDD/<emotion>/*.wav

Speech Transfer Scenarios

Scenario	Source Corpora	Target Corpus
scn1	CREMA-D, SAVEE, RAVDESS	IEMOCAP
scn2	CREMA-D, SAVEE, TESS	RAVDESS
scn3	CREMA-D, SAVEE, RAVDESS	TESS
scn4	SAVEE, RAVDESS, EmoDB	CaFE
scn5	EMOVO, ShEMO, EmoDB	AESDD
scn6	CREMA-D, SAVEE, RAVDESS	ShEMO
scn7	TESS, EmoDB	ShEMO

All scenarios use anger, happiness, sadness, disgust and fear. ShEMO has no disgust samples.

Prepare Model

For speech, place an existing VoxCeleb1 copy (development + test) under raw/voxceleb/id*/video/*.wav, then pretrain ResNet-18. The provider no longer distributes its audio.

python scripts/prepare_data.py --dataset voxceleb
python scripts/prepare_model.py

An existing encoder can be imported with --encoder /path/to/encoder.pt.

Run Experiments

From the repository root, run the Flower App:

common=(--scenario scn7 --epochs 50 --batch-size 32 --trials 10 \
        --learning-rate 0.05 --inner-lr 0.01 --momentum 0.9 --max-grad-norm 1)

# DA-MLDG
python scripts/run_experiment.py "${common[@]}" --method damldg

# Baselines
python scripts/run_experiment.py "${common[@]}" --method deepall
python scripts/run_experiment.py "${common[@]}" --method mldg
python scripts/run_experiment.py "${common[@]}" --method rsc

Note

Flower schedules independent seeded trials; the target is evaluated only after training. Stability safeguards add a gradient cap of 1 and final source-only BatchNorm calibration to every method and pretraining. Disable them with --max-grad-norm 0 --no-bn-calibration. Models and metrics are saved under results/.

TRILLsson and PACS

Prepare TRILLsson3 embeddings and run the same speech experiment:

python scripts/prepare_model.py --model trillsson --scenario scn7
python scripts/run_experiment.py "${common[@]}" --representation trillsson

The official TRILLsson3 encoder is frozen. Its 1,024-dimensional output is arranged as a 32 × 32 single-channel input to ResNet-18.

Prepare PACS and its ImageNet encoder:

python scripts/prepare_data.py --dataset pacs
python scripts/prepare_model.py --model imagenet
python scripts/run_experiment.py --pacs-target sketch --trials 10

Select art_painting, cartoon, photo or sketch as the held-out target. The remaining domains supply training data.

Use --parallel-trials to run trials concurrently, --backend sequential for the same optimizer in one process, and --help for all settings.

Citation

@inproceedings{gandhi2023damldg,
  title     = {On efficacy of Meta-Learning for Domain Generalization in Speech Emotion Recognition},
  author    = {Gandhi, Raeshak King and Tsouvalas, Vasileios and Meratnia, Nirvana},
  booktitle = {IEEE PerCom Workshops},
  year      = {2023},
  pages     = {421--426},
  doi       = {10.1109/PerComWorkshops56833.2023.10150265}
}