Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation - AI 论文深度分析

TL;DR
Mega-ASR introduces robust speech recognition using a 2.4M-clip synthetic dataset with 7 acoustic effects. Through progressive fine-tuning and dual-granularity policy optimization, it achieves 65.8%/69.

论文原始摘要

Despite rapid advances in automatic speech recognition (ASR) and large audio-language models, robust recognition in real-world environments remains limited by an "acoustic robustness bottleneck": models often lose acoustic grounding and produce omissions or hallucinations under severe, compositional distortions. We propose Mega-ASR, a unified ASR-in-the-wild framework that combines scalable compound-data construction with progressive acoustic-to-semantic optimization. We introduce Voices-in-the-Wild-2M, covering 7 classic acoustic phenomena and 54 physically plausible compound scenarios, and train Mega-ASR with Acoustic-to-Semantic Progressive Supervised Fine-Tuning and Dual-Granularity WER-Gated Policy Optimization. Extensive experiments demonstrate that Mega-ASR achieves significant advantages over prior state-of-the-art systems on adverse-condition ASR benchmarks (45.69% vs. 54.01% on VOiCES R4-B-F, and 21.49% vs. 29.34% on NOIZEUS Sta-0). On complex compositional acoustic scenarios, Mega-ASR further delivers over 30% relative WER reduction against strong open- and closed-source baselines, establishing a scalable paradigm for robust ASR in-the-wild.

Paper Collector 中文速览

Mega-ASR通过大规模声学仿真和渐进优化提升极端环境语音识别鲁棒性

方法概述

构建覆盖7种经典声学现象和54种物理可行复合场景的200万样本数据集；采用声学到语义渐进监督微调逐步提升模型鲁棒性；引入双粒度WER门控策略优化，在词级和句级两个粒度上动态调整学习策略

核心贡献

提出Mega-ASR框架，构建 Voices-in-the-Wild-2M 数据集（54种复合声学场景），设计声学到语义渐进监督微调和双粒度WER门控策略优化算法

原始来源与核验范围

核验仅覆盖论文正文中的主张、证据片段和参考文献关系。NGJOO 未独立运行作者代码、重做实验或验证真实部署效果；页面中的可复现性分数是文档完整度评估，不是复现实验结果。

已证实

证据不足

无法验证

N/A

可复现性

置信度

74%

核心问题

How can automatic speech recognition systems be made robust to severe, compositional real-world acoustic distortions where existing models routinely fail with WER exceeding 30%?

核心方法

The authors construct VOICES-IN-THE-WILD-2M through spectrogram-level simulation with 7 atomic acoustic effects (noise, far-field, obstructed, echo&reverb, recording, electronic distortion, transmission dropout) composed into 54 compound scenarios. Mega-ASR is trained via Acoustic-to-Semantic Progressive Supervised Fine-Tuning with WER-graded curriculum expanding from WER<30% to WER<70%, followed by Dual-Granularity WER-Gated Policy Optimization that applies token- and sentence-level rewards based on error regime.

方法组件

The router is a lightweight audio-quality classifier predicting clean vs degraded speech using a single-layer Transformer architecture.
Input processing extracts 80-dimensional log-Mel features from the waveform.
A convolutional frontend performs temporal downsampling before Transformer encoding.
Attention-pooling aggregates frame-level representations into utterance-level embeddings.
A linear binary classification head produces the final routing decision.

论点验证

证据不足 (60%) we introduce VOICES-IN-THE-WILD-2M, a large-scale dataset built through spectrogram-level code-based simulation, the design choice that makes its scale tractable

已证实 (85%) we first identify, as shown in Figure 2, seven classic in-the-field acoustic effects {noise, far-field, obstructed, echo&reverb, recording, electronic distortion, transmission dropout}, which we term atomic acoustic effects

证据不足 (55%) The atomic phenomena are then composed into 54 agent-validated configurations, yielding 2.4M synthesized clips whose effectiveness on real-world data is empirically verified after mixed-condition training
The paper states 54 configurations and 2.4M clips, but 'agent-validated' is not explained - no description of what agents were used, how validation was performed, or validation criteria. The 'effectiveness on real-world data is empirically verified'

已证实 (80%) even the state-of-the-art Qwen3-ASR attains a high average WER of 35% on this benchmark

证据不足 (40%) We adopt the latter for its flexibility and, more importantly, its scalability

证据不足 (50%) we simulate each of the seven phenomena directly on the spectrogram via filtering, convolution, and related signal-level transformations, with parameters iteratively tuned to best fit real-world recordings

已证实 (75%) we scale from atomic effects to compound scenarios by composing 2 to 5 atomic effects, retaining only physically plausible combinations

已证实 (70%) as shown in Figure 3, the Linear distribution is adopted as the severity profile of the dataset

已证实 (80%) We further release Voices-in-the-wild-Bench, a 5,000-clip English/Mandarin evaluation set covering the same seven atomic phenomena as VOICES-IN-THE-WILD-2M, comprising 3,500 synthetic clips and 1,500 real-world recordings collected from internet sources and 16 human participants
The paper provides specific details about the benchmark: 5,000 clips, English/Mandarin, 3,500 synthetic + 1,500 real-world recordings from internet sources and 16 human participants. This is a factual description of a released resource.

已证实 (85%) We propose a framework, as shown in figure 4 for robust speech recognition under complex acoustic conditions

已证实 (80%) We first develop Mega-ASR-Base on top of Qwen3-ASR via Acoustic-to-Semantic Progressive Supervised Fine-Tuning, instilling perceptual robustness and semantic recovery

已证实 (85%) We then apply Dual-Granularity WER-Gated Policy Optimization that supplies token- and sentence-level rewards, dynamically modulating their granularity to mitigate WER reward failure
DG-WGPO is described in detail with mathematical formulations for token-level and sentence-level rewards, WER-gated fusion, and the complete reward function. The methodology is well-specified and reproducible.

证据不足 (50%) We observe that existing ASR models struggle to maintain reliable acoustic understanding in the medium and high WER regimes, often producing empty outputs, severe hallucinations, or off-audio transcriptions

已证实 (85%) A2S-SFT addresses them in three phases: (i) a WER-graded curriculum on the encoder and aligner, successively expanding from WER<30% to WER<50% and finally to WER<70%, to build acoustic perception incrementally; (ii) LLM fine-tuning on full WER<70% samples to activate semantic recovery under unreliable acoustic evidence; and (iii) joint fine-tuning of encoder, aligner, and LLM for end-to-end alignment
The three-phase A2S-SFT training procedure is described in detail with specific WER thresholds, trainable components, and hyperparameters. The methodology is well-specified and reproducible.

证据不足 (45%) We observe during training that errors when WER<=30% are predominantly word-level confusions, whereas beyond this threshold they shift abruptly into sentence-level failures such as hallucinations and omissions

证据不足 (40%) The standard WER reward, however, conflates these two regimes and further saturates under heavy degradation, collapsing intra-group dispersion precisely where the policy needs it most

已证实 (85%) We therefore propose Dual-Granularity WER-Gated Policy Optimization (DG-WGPO), which retains a classic static rule-based reward (WER plus a repetition penalty) as the basic learning signal, and introduces a Dual-Granularity Dynamic Reward as its core, applying WER-gated fine- and coarse-grained rewards aligned with the two error regimes
DG-WGPO is fully specified with mathematical formulations for static rewards (WER + repetition penalty), token-level refinement reward, sentence-level reconstruction reward, and WER-gated fusion. The contribution is well-documented and reproducible.

已证实 (90%) We set the three hyperparameters as τ = 0.3, α s = 0.4, and α dyn = 0.6
The hyperparameter values are explicitly stated in p_21. This is a factual design choice specification.

证据不足 (55%) Training MEGA-ASR on heavily degraded audio sharpens its noise robustness but partially erodes complementary capabilities such as clean-speech recognition, hotword recognition, and streaming ASR

已证实 (85%) we fine-tune a lightweight binary classifier with LoRA on a mixture of clean speech and Voices-in-the-Wild samples, predicting whether an input requires Mega-ASR's noise-robust weights or the original backbone
The router implementation is described in detail: lightweight binary classifier with LoRA, trained on clean speech (LibriSpeech, AISHELL-1, CommonVoice22, WenetSpeech) and degraded speech (VOICES-IN-THE-WILD-2M). The methodology is well-specified.

... 共 53 个论点

可复现性评估

较低可复现性 (0%)

缺失的复现细节

Code is not publicly available
Training data (VOICES-IN-THE-WILD-2M) is not publicly available
Evaluation benchmark (Voices-in-the-Wild-Bench) is not publicly available
Random seeds not specified
Hardware specifications not provided (GPU type, number of GPUs, memory requirements)
Batch size not specified for SFT or RL stages
Optimizer type and configuration (weight decay, betas, etc.) not mentioned
SFT training duration (steps/epochs) not specified
Dynamic routing LoRA architecture details missing (rank, alpha, target modules)
Router training procedure and data not described

局限与证据边界

Training MEGA-ASR on heavily degraded audio sharpens its noise robustness but partially erodes complementary capabilities such as clean-speech recognition, hotword recognition, and streaming ASR

本分析由 PDF 阅读助手自动生成，仅供参考，不构成学术评审意见。验证结论和可复现性评估基于论文文本自动分析，可能存在偏差。原始论文请参阅 arXiv。

分析时间：2026-05-22T01:30:26+00:00 · 数据来源：Paper Collector