Darwin introduces diagnostic-guided evolutionary merging for training-free scaling of language model reasoning. Using MRI diagnostics and a 14-dimensional genome with MRI-Trust Fusion, Darwin-27B-Opus achieves 86.9% on GPQA Diamond (rank #6 of 1,252), outperforming its parent by +1.
论文原始摘要
We present Darwin Family, a framework for training-free evolutionary merging of large language models via gradient-free weight-space recombination. We ask whether frontier-level reasoning performance can be improved without additional training, by reorganizing latent capabilities already encoded in existing checkpoints. Darwin introduces three key ideas: (i) a 14-dimensional adaptive merge genome enabling fine-grained component- and block-level recombination; (ii) MRI-Trust Fusion, which adaptively balances diagnostic layer-importance signals with evolutionary search through a learnable trust parameter; and (iii) an Architecture Mapper that enables cross-architecture breeding between heterogeneous model families. Empirically, the flagship Darwin-27B-Opus achieves 86.9% on GPQA Diamond, ranking #6 among 1,252 evaluated models, and outperforming its fully trained foundation model without any gradient-based training. Across scales from 4B to 35B parameters, Darwin models consistently improve over their parents, support recursive multi-generation evolution, and enable a training-free evolutionary merge that combines Transformer- and Mamba-based components. Together, the Darwin Family demonstrates that diagnostic-guided evolutionary merging is a practical and reproducible alternative to costly post-training pipelines for reasoning-centric language models.
Paper Collector 中文速览
训练自由的进化式模型融合框架
方法概述
采用梯度无关的权重空间重组策略，通过14维基因组实现细粒度组件和层级融合；MRI-Trust融合机制平衡诊断层重要性与进化搜索；架构映射器支持Transformer与Mamba等异构模型家族间的跨架构繁衍。
核心贡献
提出Darwin框架，通过14维自适应融合基因组和MRI-Trust信任融合，实现无需训练的进化式模型合并，显著提升推理能力。
原始来源与核验范围
核验仅覆盖论文正文中的主张、证据片段和参考文献关系。NGJOO 未独立运行作者代码、重做实验或验证真实部署效果；页面中的可复现性分数是文档完整度评估，不是复现实验结果。
核心问题
Can diagnostic-guided evolutionary merging improve language model reasoning performance without gradient-based training, and how does it compare to static merging and evolutionary merging without diagnostics?
核心方法
Darwin combines Model-layer Response Importance (MRI) diagnostic priors with a 14-dimensional genome encoding merge configurations, reconciled through MRI-Trust Fusion with learnable trust parameter τ. A two-phase evolutionary optimization strategy separates structural screening from empirical evaluation, operating entirely in weight space without gradient-based training.
方法组件
- Static merging uses fixed coefficients like weight averaging or task arithmetic.
- Performance degrades when merging heterogeneous specialists due to representational interference.
- Recent advances use structured sparsification and selective parameter alignment.
- Training-free merging is a viable alternative to expensive multi-task training pipelines.
- Evolutionary optimization enables gradient-free exploration of merge configurations.
- Neuroevolution demonstrates effective optimization of high-dimensional neural architectures.
- Evolutionary search can discover merging recipes that outperform manual heuristics.
- Existing evolutionary merging methods lack diagnostic guidance.
- Hybrid models combine attention transformers with state-space models for improved efficiency.
- Complementary inductive biases can be combined within a single model.
论点验证
MRI is formally introduced in p_19-21 with a complete mathematical formulation combining static tensor statistics and probe-based functional responses. The formula is specified, and its role as a diagnostic prior is clearly explained.
The Architecture Mapper is described in detail in p_22-23 with a specific compatibility score formula and explanation of how it establishes tensor-level correspondences through constrained greedy matching.
Specific numerical results are provided in p_30: 86.9% accuracy on GPQA Diamond, #6 ranking among 1,252 models, and outperforming the strongest parent. These are concrete, verifiable claims with specific numbers.
Specific numerical claims are provided in p_39: #6 ranking on GPQA Diamond Leaderboard and +1.4pp improvement over Father Qwen3.5-27B. These are concrete, verifiable results.
Specific numerical values are provided in p_33: τ converges to intermediate values (τ ≈ 0.35-0.55 across scales). This is concrete quantitative evidence about the learned parameter behavior.
Specific numerical result is provided in p_36: +1.2pp improvement for τ=1 (static MRI) relative to τ=0 (genome-only). This is a concrete ablation result with specific numbers.
Specific numerical result is provided in p_36: evolved τ variant achieves +0.9pp gain over fixed τ=0.7. This is a concrete ablation result with specific numbers.
Specific numerical result is provided in p_36: +2.5pp improvement for full adaptive variant over no-MRI baseline on GPQA. This is a concrete ablation result with specific numbers.
Specific numerical data is provided in p_80: 'community downloads exceed 96,000 across official and quantized distributions' with reference to Table 7. This is a concrete, verifiable metric.
The paper explicitly states in p_21 that α is fixed to 0.5 in all experiments. This is a clear design choice specification.
The paper explicitly states the coefficient values in p_23: β_1 = 0.5, β_2 = 0.3, β_3 = 0.2. This is a clear design choice specification.
... 共 43 个论点
可复现性评估
较低可复现性 (0%)
缺失的复现细节
- Evolutionary algorithm configuration (algorithm type, population size, number of generations, mutation/crossover rates, selection mechanism)
- Genome encoding scheme for merge configurations - exact representation and dimensionality
- Complete MRI computation formulas - detailed specification of static term (entropy, variance, ℓ2-norm) and probe term calculations
- MRI-Trust Fusion mechanism - exact formula for combining MRI-derived and genome-derived ratios
- Calibration set specification (size, content, source dataset) for probe-based diagnostics
- DARE-TIES merge kernel parameters (drop rate, rescaling factors)
- Compatibility threshold value for Architecture Mapper greedy matching
- Random seeds for evolutionary search and evaluation
- Hardware/environment specifications (GPU type, memory, framework versions)
- Number of stochastic decoding runs and exact inference settings (temperature, top-p, etc.)
局限与证据边界
- In cases where both parent models exhibit highly similar capabilities and error patterns, evolutionary merging provides limited benefit. When neither parent contributes a distinct or dominant capability, recombination primarily redistributes redundant structure rather than composing complementary functions, resulting in negligible or no improvement.
- Some non-improving merges involve parent models whose internal representations are poorly aligned, even when nominally derived from the same base architecture. In such cases, weight-space recombination may disrupt reasoning-critical pathways faster than evolutionary optimization can recover them, leading to early saturation of gains.
- Darwin relies on MRI as a soft diagnostic prior rather than a ground-truth indicator. When diagnostic signals are weak, noisy, or inconsistent across layers—for example, when reasoning-relevant activations are diffused rather than localized—MRI guidance becomes less informative.
- Some parent pairs already approach a local optimum for the targeted reasoning benchmarks. In these regimes, Darwin's evolutionary search converges quickly, but further improvement is constrained by the lack of latent complementary structure rather than by search inefficiency.
- Darwin is most effective when applied to heterogeneous but compatible parent models with partially complementary reasoning structure.
本分析由 PDF 阅读助手 自动生成，仅供参考，不构成学术评审意见。验证结论和可复现性评估基于论文文本自动分析，可能存在偏差。原始论文请参阅 arXiv。
分析时间：2026-05-18T07:17:29+00:00 · 数据来源：Paper Collector