ViProDID (Hierarchical-DialectMoE)
Speaker-Disjoint 63-Province Vietnamese Dialect Identification with Prosodic Gating
Official implementation of ViProDID for speaker-disjoint fine-grained Vietnamese dialect identification across all 63 provinces and 3 macro-regions on ViMD, combining `wav2vec2-large-vi` with a 12-dimensional prosodic gating mechanism (59.32% province Macro-F1, 94.42% region Macro-F1).
The Technical Bottleneck
Identifying a speaker's exact province across all 63 Vietnamese provinces—without speaker overlap between training and testing—is challenging because neighboring provinces share lexical items while differing in subtle pitch contours, energy dynamics, and spectral tilt.
Why Existing Baselines Fail
Self-supervised speech backbones like wav2vec2 capture phonetic content well but can under-utilize explicit suprasegmental prosody (F0 pitch and energy contours) that characterizes Vietnamese tonal dialects.
Core Formulation & Insight
ViProDID extracts contextual acoustic representations from `wav2vec2-large-vi` and fuses them with 12 explicit prosodic features (pitch, energy, spectral descriptors) through a learnable gating mechanism, comparing flat and hierarchical Mixture-of-Experts (MoE) routing across 3 regions and 63 provinces.
Architectural Design
Acoustic-prosodic fusion network pairing `wav2vec2-large-vi` with a 12D prosodic gating module and hierarchical region-to-province classification.
1. Speaker-Disjoint ViMD Partitioning
Enforces strict speaker-disjoint splits across all 63 provinces and 3 macro-regions (North, Central, South) to prevent speaker-identity shortcut learning.
2. Dual Acoustic & 12D Prosodic Extraction
Extracts frame-level embeddings from `wav2vec2-large-vi` alongside a 12-dimensional prosodic vector capturing pitch (F0), energy, and spectral statistics.
3. Learnable Prosodic Gating & Hierarchical MoE
Modulates acoustic representations via a prosodic gate and evaluates hierarchical region-conditioned Mixture-of-Experts classification.
Technical Stack & Systems Engineering
Implementation Highlights
- →Implemented end-to-end prosodic feature extraction, gated acoustic fusion, and paired-bootstrap significance testing across 63 provincial classes.
Datasets & Protocol
Datasets & Corpora
- ViMD (Vietnamese Multi-Dialect Dataset — 63 provinces, 3 regions, speaker-disjoint split)
Evaluation Protocol
Evaluated acoustic-only baselines against prosodic-gated and hierarchical MoE architectures with paired-bootstrap confidence intervals.
Tracked Evaluation Metrics
Achieved 59.32 ± 0.18% Macro-F1 across 63 provinces and 94.42 ± 0.15% Macro-F1 across 3 regions (paired-bootstrap p < 0.001).
Speaker-disjoint evaluation on ViMD (p < 0.001)
North / Central / South macro-dialect classification
Pitch, energy, and spectral descriptors fused via gating
Empirical Analysis & Interpretation
Explicit prosodic gating provides a statistically significant boost (p < 0.001) on fine-grained 63-province classification under strict speaker-disjoint evaluation.
What This System Contributes
- 01Developed the full ViProDID (`Hierarchical-DialectMoE`) training, prosodic extraction, and statistical evaluation repository.
Boundary Conditions & Future Work
- ·Manuscript is currently submitted and under peer review (explicitly marked as Under Review, not claimed as accepted).
ViProDID: Speaker-Disjoint Fine-Grained Vietnamese Dialect Identification across 63 Provinces via Prosodic Gating and Hierarchical Mixture-of-Experts
Van-Truong Le