SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer - AI 论文深度分析

TL;DR
SANA-WM generates minute-long 720p videos with precise 6-DoF camera control using hybrid attention and dual-rate conditioning, reducing memory from 8.9 to 5.4 GB and latency 3.4× while maintaining quality on a 213K-clip corpus.

论文原始摘要

We introduce SANA-WM, an efficient 2.6B-parameter open-source world model natively trained for one-minute generation, synthesizing high-fidelity, 720p, minute-scale videos with precise camera control. SANA-WM achieves visual quality comparable to large-scale industrial baselines such as LingBot-World and HY-WorldPlay, while significantly improving efficiency. Four core designs drive our architecture: (1) Hybrid Linear Attention combines frame-wise Gated DeltaNet (GDN) with softmax attention for memory-efficient long-context modeling. (2) Dual-Branch Camera Control ensures precise 6-DoF trajectory adherence. (3) Two-Stage Generation Pipeline applies a long-video refiner to stage-1 outputs, improving quality and consistency across sequences. (4) Robust Annotation Pipeline extracts accurate metric-scale 6-DoF camera poses from public videos to yield high-quality, spatiotemporally consistent action labels. Driven by these designs, SANA-WMdemonstrates remarkable efficiency across data, training compute, and inference hardware: it uses only $\sim$213K public video clips with metric-scale pose supervision, completes training in 15 days on 64 H100s, and generates each 60s clip on a single GPU; its distilled variant can be deployed on a single RTX 5090 with NVFP4 quantization to denoise a 60s 720p clip in 34s. On our one-minute world-model benchmark, SANA-WM demonstrates stronger action-following accuracy than prior open-source baselines and achieves comparable visual quality at $36\times$ higher throughput for scalable world modeling.

Paper Collector 中文速览

2.6B参数高效世界模型，实现分钟级720p视频生成

方法概述

结合Gated DeltaNet与softmax的混合线性注意力处理长上下文；双分支架构控制6自由度相机轨迹；两阶段生成（基础模型+长视频精炼器）；从公共视频提取度量级相机姿态构建高质量训练数据。

核心贡献

提出SANA-WM，通过混合线性注意力、双分支相机控制、两阶段生成管线和鲁棒标注管道，实现高效分钟级世界建模，在单GPU上生成60秒720p视频。

原始来源与核验范围

核验仅覆盖论文正文中的主张、证据片段和参考文献关系。NGJOO 未独立运行作者代码、重做实验或验证真实部署效果；页面中的可复现性分数是文档完整度评估，不是复现实验结果。

已证实

证据不足

无法验证

N/A

可复现性

置信度

81%

核心问题

How can we generate minute-long 720p videos with precise 6-DoF camera control under strict computational efficiency constraints?

核心方法

SANA-WM combines three designs: hybrid GDN/softmax attention replacing every fourth block with softmax for efficient long-context modeling, dual-rate camera conditioning using latent-frame UCPE for global pose and raw-frame Plücker mixing for intra-stride motion, and a second-stage refiner with truncated-σ flow matching. Progressive four-stage training uses LTX2-VAE for spatiotemporal compression and self-forcing distillation to reduce sampling to four steps.

方法组件

SANA-WM targets minute-long high-resolution video generation with precise camera control under efficiency constraints.
Three challenges addressed are compute costs, camera conditioning accuracy, and visual quality degradation.
The method builds on SANA-Video with hybrid attention, dual-rate conditioning, and a second-stage refiner.
Cumulative linear attention maintains constant memory but lacks decay mechanisms, causing drift in long sequences.
Gated DeltaNet adds decay gates and delta-rule corrections to the recurrent state.
Key scaling with 1/√DS ensures stable convergence while other scaling methods cause gradient instability.
Hybrid GDN/softmax attention replaces every fourth block with softmax attention for improved generation quality.

论点验证

已证实 (85%) SANA-WM is a world model designed to generate minute-long high-resolution videos with precise camera control under strict efficiency constraints.
The paper provides quantitative evidence for all key aspects: 60-second video generation (p_5, p_26, p_48), 720p resolution (p_5, p_26, p_48), camera control evaluation (p_55 with specific metrics like ATE, RPE, RotErr), and efficiency metrics (p_26

已证实 (90%) SANA-WM builds on SANA-Video with three complementary designs: a hybrid GDN/softmax attention architecture for efficient long-context modeling, dual-rate camera conditioning for coarse-to-fine trajectory control, and a second-stage refiner for minute-length video to improve fidelity.
The three designs are explicitly stated in p_5 and elaborated in subsequent sections: hybrid GDN/softmax (p_14), dual-rate camera conditioning (p_15-20), and second-stage refiner (p_21). Each design is fully specified with implementation details.

证据不足 (50%) We train progressively from short clips to minute-scale videos, increasing sequence length and introducing architectural components in four stages.

已证实 (90%) To make minute-scale video modeling computationally feasible, we replace the baseline VAE with LTX2-VAE to leverage its superior spatiotemporal compression ratio.
The VAE replacement is justified with quantitative evidence in p_27: LTX2 cuts peak memory from 8.9 to 5.4 GB and latency 3.4× while being quality-neutral (+0.0012 Total). This directly supports the computational feasibility claim.

已证实 (85%) We use self-forcing distillation to reduce sampling to four denoising steps.
p_7 explicitly states self-forcing distillation reduces sampling to four denoising steps. The reference [27] is cited. However, no ablation comparing different step counts is shown to validate this specific choice.

已证实 (80%) For deployment, we add attention-sink tokens and local temporal windows to the softmax attention layers, keeping softmax memory and per-chunk latency constant with respect to rollout length.
p_7 and p_26 describe attention-sink tokens and local temporal windows. p_26 explicitly states this keeps memory constant. However, no ablation comparing with/without this design is provided.

已证实 (85%) This compact state has no explicit decay or saliency mechanism: stale features accumulate with the same effective weight as more recent ones. At the minute scale, the unbounded growing state causes drift and degrades training stability.

已证实 (85%) To enhance long-video generation performance, we further fine-tune the GDN model by replacing every fourth block with standard softmax attention, while retaining the original QKV and output projections.
p_14 explicitly describes replacing every fourth block with softmax attention. p_27 shows the hybrid 15-GDN/5-softmax backbone raises Total to 0.853 while keeping memory at 5.7 GB, providing quantitative validation.

已证实 (85%) We use dual-rate geometric conditioning: latent-frame UCPE captures global 6-DoF pose, while raw-frame Plücker mixing compensates motion inside each VAE stride.
p_15 explicitly states the dual-rate design. p_16-20 provide full implementation details for both UCPE (coarse) and Plücker mixing (fine) branches. Tab. 4 (p_27) shows this combination gives lowest Pi3X errors.

已证实 (80%) Following LTX-Video, we add a second-stage refiner to improve stage-1 SANA-WM visual fidelity.
p_21 states the refiner following LTX-Video. Implementation details are provided. However, the justification 'following LTX-Video' is an appeal to prior work rather than an ablation showing this specific choice is optimal for SANA-WM.

已证实 (90%) We use truncated-σ flow matching: x_ℓ is perturbed with a large starting noise (σ_start = 0.9), and the model learns to map this noisy source toward x_h, encouraging refinement over full reconstruction.
p_21 and p_30-37 provide complete mathematical formulation of truncated-σ flow matching with σ_start = 0.909375. The rationale (encouraging refinement over reconstruction) is clearly explained.

已证实 (85%) We build a robust annotation pipeline that re-annotates seven open-source video sources with metric-scale camera poses, yielding a 213K-clip corpus.
p_22 states the 213K-clip corpus from seven sources. Fig. 3 and Tab. 1 are referenced for details. The pipeline is described in p_23-25. However, Tab. 1 is not shown in the provided text, so the exact breakdown cannot be verified.

已证实 (75%) We replace the depth backend with Pi3X and MoGe-2: Pi3X provides long-sequence-consistent depth, while MoGe-2 provides accurate per-frame metric scale.
p_23 describes replacing VIPE's depth backend with Pi3X and MoGe-2 with specific rationale. However, no ablation comparing different depth backends is provided to validate this specific combination is optimal.

已证实 (80%) We adapt VIPE to enable per-frame intrinsic optimization for more robust internet video annotation.
p_23 and p_43 describe the per-frame intrinsic optimization extension to VIPE. The implementation is specified (storing as (N,V,D) tensor). However, no quantitative comparison showing improved accuracy is provided.

已证实 (80%) We augment static 3D scene data with rendered camera trajectories. DL3DV contains static 3D captures rather than native one-minute videos, so we fit one FCGS 3D Gaussian Splatting reconstruction per scene, design diverse one-minute camera paths, and render long videos with known intrinsics and extrinsics.
p_24 describes the DL3DV augmentation pipeline with FCGS reconstruction, trajectory design, and rendering. The process is well-specified. However, no analysis of the quality of these synthetic videos is provided.

已证实 (70%) We refine the rendered videos with DiFix3D to reduce splatting artifacts.
p_24 mentions DiFix3D refinement to reduce splatting artifacts with reference to App. B.2. The design choice is stated but not validated with visual examples or quantitative artifact metrics in the main text.

已证实 (90%) LTX2 is quality-neutral (+0.0012 Total) but cuts peak memory from 8.9 to 5.4 GB and latency 3.4×.
p_27 provides exact quantitative results from Tab. 3: +0.0012 Total score, memory 8.9→5.4 GB, latency 3.4× reduction. These are specific, verifiable numbers from controlled ablation.

已证实 (90%) The hybrid 15-GDN/5-softmax backbone raises Total to 0.853 while keeping memory at 5.7 GB.
p_27 provides exact quantitative results: Total 0.853, memory 5.7 GB. These are specific numbers from the ablation study.

已证实 (90%) Our 1/√DS scaling is the only variant that ensures stable convergence. In contrast, both the L2 (1/√D) and no-scale baselines suffer from immediate gradient instability, triggering NaN events at steps 16 and 1, respectively.
p_27 references Fig. 6 showing that 1/√DS scaling ensures stable convergence while L2 and no-scale baselines trigger NaN events at steps 16 and 1 respectively. This is strong empirical evidence from controlled comparison.

已证实 (85%) Input-level Plücker gives only small gains, attention-level PRoPE and UCPE greatly improve control, and the dual UCPE + Plücker mixing branch gives the lowest Pi3X errors with competitive FVD.
p_27 summarizes Tab. 4 results: Plücker gives small gains, PRoPE/UCPE greatly improve control, dual UCPE+Plücker gives lowest Pi3X errors with competitive FVD. Specific metrics from controlled ablation on OmniWorld validation split.

... 共 34 个论点

可复现性评估

较低可复现性 (0%)

缺失的复现细节

Complete model architecture specifications (number of layers, hidden dimensions, number of attention heads, total blocks)
Training hyperparameters (learning rate, batch size, epochs, optimizer, weight decay, learning rate schedule)
GDN-specific parameters (decay gate initialization, delta-rule parameters, key scaling factors)
Dual-rate camera conditioning implementation details and architecture
Second-stage refiner architecture, training procedure, and integration with base model
Training data specifications (dataset sources, size, preprocessing steps, train/val splits)
Hardware specifications and training duration/compute requirements
Random seeds for reproducibility
VAE/encoder architecture and latent space dimensions
Evaluation metrics implementation details and benchmark specifications

局限与证据边界

This compact state has no explicit decay or saliency mechanism: stale features accumulate with the same effective weight as more recent ones. At the minute scale, the unbounded growing state causes drift and degrades training stability.
SANA-WM remains scale-limited, lacks explicit 3D scene memory, and can drift in dynamic scenes, rare viewpoints, or longer rollouts.
Directly finetuning the distilled few-step refiner was unstable in our experiments.
Generated videos may be mistaken for real observations if provenance is not documented.

本分析由 PDF 阅读助手自动生成，仅供参考，不构成学术评审意见。验证结论和可复现性评估基于论文文本自动分析，可能存在偏差。原始论文请参阅 arXiv。

分析时间：2026-05-17T13:25:55+00:00 · 数据来源：Paper Collector