ViProDID: Speaker-Disjoint Fine-Grained Vietnamese Dialect Identification across 63 Provinces via Prosodic Gating and Hierarchical Mixture-of-Experts
1University of Science, Vietnam National University Ho Chi Minh City (VNU-HCM) · SELab
Manuscript Submitted for Peer Review (Official Implementation Available)
Statistically significant improvement (paired-bootstrap p < 0.001) on speaker-disjoint ViMD
High macro-regional separability across North, Central, and South dialects
Paper Abstract
Fine-grained dialect identification across all 63 provinces of Vietnam under strict speaker-disjoint splits poses severe acoustic and prosodic challenges due to subtle intra-regional tonal variations. ViProDID fuses pretrained Vietnamese wav2vec2 representations (`wav2vec2-large-vi`) with 12-dimensional explicit pitch, energy, and spectral prosodic features through a learnable prosodic gating mechanism and evaluates hierarchical Mixture-of-Experts (MoE) routing across 3 macro-regions and 63 provinces on the ViMD dataset.
Research Motivation
Most Vietnamese dialect systems only classify 3 coarse macro-regions (North, Central, South) or leak speaker identity across train/test splits. Distinguishing all 63 provinces under speaker-disjoint evaluation requires fusing phonetic representations with explicit tonal/prosodic contours.
Guiding Research Questions
- RQ1How effectively does gating `wav2vec2-large-vi` representations with 12-dimensional pitch, energy, and spectral prosodic features improve 63-province speaker-disjoint classification?
Methodology & Architecture
Extracts acoustic embeddings from `wav2vec2-large-vi` alongside 12-dimensional explicit prosodic descriptors (F0 pitch statistics, energy dynamics, spectral tilt) and fuses them via a prosodic gating module and hierarchical region-to-province MoE architecture on the speaker-disjoint ViMD benchmark.
Audio Waveform → Parallel `wav2vec2-large-vi` Acoustic Backbone + 12D Prosodic Feature Extractor (Pitch/Energy/Spectral) → Learnable Prosodic Gating Fusion → Hierarchical 3-Region / 63-Province Prediction.
- · ViMD (Vietnamese Multi-Dialect Dataset — 63 provinces, 3 macro-regions, strict speaker-disjoint split)
63-Province Macro-F1 · 3-Region Macro-F1 · Paired-Bootstrap Significance (p-value)
Primary Contributions
- 01Developed ViProDID (`Hierarchical-DialectMoE`) for speaker-disjoint 63-province Vietnamese dialect identification.
- 02Combined `wav2vec2-large-vi` with a 12-dimensional prosodic gating mechanism, achieving 59.32% province Macro-F1 and 94.42% region Macro-F1.
Analyzed Boundary Conditions
- ·Currently under peer review; province-level confusion remains highest among geographically adjacent provinces with shared tonal inventories.
BibTeX Entry
@misc{le2026viprodid,
author = {Le, Van-Truong},
title = {ViProDID: Fine-Grained Vietnamese Dialect Identification with Prosodic Gating and Hierarchical Mixture of Experts},
year = {2026},
howpublished = {\url{https://github.com/lee-vtruong/Hierarchical-DialectMoE}},
note = {Manuscript submitted for publication}
}