Multilingual Fact-Checking Retrieval Pipeline
Ensemble BiLSTM-Attention Bi-Encoders & Dense Hard-Negative Cross-Encoder Reranking
A two-stage multilingual and cross-lingual claim verification retrieval system evaluated on SemEval-2025 Task 7 across 10 languages. Achieves 0.9163 Overall S@10 and 0.8833 Cross-Lingual S@10 (+8.0% gain from reranking), surpassing the competition winner in cross-lingual retrieval.
The Technical Bottleneck
Verifying social media misinformation requires matching noisy, multimodal posts (with OCR text) against fact-checked claims across 10+ languages. Standard dense retrievers struggle when claims are topically identical but factually opposite, especially in zero-shot cross-lingual transfers (e.g., Polish and Turkish).
Why Existing Baselines Fail
While two-stage retrieve-then-rerank pipelines are standard in monolingual IR, their behavior under cross-lingual fact-checking ambiguity is highly sensitive to pooling architecture and negative sampling. We investigated why naive BM25 hard negative mining fails and how retriever-aligned dense hard negatives unlock cross-encoder discrimination.
Core Formulation & Insight
Stage 1 replaces standard mean pooling on `multilingual-e5-large-instruct` with a sequential BiLSTM-Attention pooling head and ensembles 5 independently seeded models via FAISS. Stage 2 mines the top-10 highest-ranked false positives directly from Stage 1's output distribution as hard negatives to fine-tune a `BAAI/bge-reranker-v2-m3` Cross-Encoder.
Architectural Design
Two-stage retrieve-then-rerank architecture coupling a 5× BiLSTM-Attention E5-Large dense ensemble with a dense-hard-negative-trained BGE-M3 cross-encoder.
Two-stage multilingual fact-checking retrieval architecture (KES 2026 / Procedia Computer Science).
1. BiLSTM-Attention Token Pooling
Processes contextual token embeddings from `multilingual-e5-large-instruct` through a bidirectional LSTM (hidden dim 512) and learned attention weights α_t to emphasize sparse factual tokens.
2. 5-Seed Ensemble & FAISS Candidate Generation
Averages similarity scores across 5 diversely initialized Bi-Encoders trained with MultipleNegativesRankingLoss (τ = 0.05) to retrieve the Top-100 candidate claims.
3. Retriever-Aligned Dense Hard Negative Mining
Extracts the highest-scoring non-relevant claims from Stage 1 (Top-10 hard negatives per query), forcing the reranker to resolve semantic near-misses rather than lexical overlap.
4. BGE-M3 Cross-Encoder Reranking
Jointly encodes `[CLS] query [SEP] claim [SEP]` pairs (max length 512) through `BAAI/bge-reranker-v2-m3` (568M parameters) to produce the final Top-10 ranking.
Technical Stack & Systems Engineering
Implementation Highlights
- →Implemented custom PyTorch BiLSTM-Attention pooling layer integrated into Sentence-Transformers training loops with FP16 mixed precision and gradient checkpointing.
- →Evaluated across 19,956 train, 2,259 dev, and 4,959 test pairs covering 8 training languages (Arabic, English, French, German, Malay, Portuguese, Spanish, Thai) and 2 unseen zero-shot test languages (Polish, Turkish).
Datasets & Protocol
Datasets & Corpora
- SemEval-2025 Task 7 Multilingual & Cross-Lingual Fact-Checked Claim Retrieval Dataset (10 languages, 4,959 test queries)
Evaluation Protocol
Benchmarked against SemEval-2025 organizer baselines (BM25, Multilingual-E5-Large) and top leaderboard systems (PINGAN AI, RACAI, TIFIN India), plus controlled ablations over reranker backbones (MiniLM-L12, XLM-RoBERTa, BGE-M3) and negative sampling strategies (Random, BM25-mined, Dense-mined).
Tracked Evaluation Metrics
Achieved 0.8833 Cross-Lingual S@10 (+8.0% absolute gain over dense retrieval and +2.46% over 1st-place PINGAN AI) and 0.9163 Overall S@10.
Surpasses competition winner PINGAN AI (0.8587) by +2.46% absolute
Monolingual S@10 = 0.9215; Stage-1 Ensemble alone = 0.8845
Substantially bridges cross-lingual retrieval degradation
Demonstrates catastrophic misalignment of lexical hard negatives
Empirical Analysis & Interpretation
Our negative-sampling ablation reveals why naive reranking often fails in fact-checking: BM25-mined hard negatives share surface words in the same language and collapse cross-lingual reranking to 0.2015 S@10, whereas Stage-1 dense hard negatives expose the Cross-Encoder to cross-lingual semantic near-misses.
What This System Contributes
- 01First author: designed the BiLSTM-Attention ensemble and dense hard-negative reranking pipeline, conducted the SemEval-2025 Task 7 experiments, and authored the KES 2026 paper.
Boundary Conditions & Future Work
- ·Cross-Encoder inference scales linearly with candidate depth K; future work explores distilling cross-encoder scores into lightweight late-interaction models.
Enhancing Multilingual Fact-Checking Retrieval with Ensemble Bi-Encoders and Architecture-Aware Hard Negative Reranking
Van-Truong Le, Duy-Minh Nguyen-Tran, Tung Le, Huy Tien Nguyen