TL;DR
RLDX-1 is a Vision-Language-Action model for human-like dexterous manipulation that integrates motion awareness, long-term memory, and physical sensing beyond versatile intelligence.
36
已证实
11
证据不足
0
无法验证
N/A
可复现性
置信度
76%

核心问题

How can Vision-Language-Action models be enhanced with motion awareness, long-term memory, and physical sensing capabilities to achieve human-like dexterous manipulation in real-world environments?

核心方法

{'approach': 'RLDX-1 combines four components: a unified neural architecture with a temporally-aware VLM and Multi-Stream Action Transformer (MSAT) processing cognition features, proprioceptive states, and physical signals through dedicated streams; a synthetic data generation pipeline using video generative models with motion-consistency filtering; a three-stage training procedure (pre-training, mid-training, post-training) progressively specializing from generalist to task-specific policies; and inference optimization through static graph conversion and custom kernel fusion.', 'key_components': ['The architecture supports diverse functionalities through effective processing of heterogeneous inputs.', 'Two main components comprise the architecture: a temporally aware VLM and a multimodal action model.', 'RLDX-1-VLM is built on Qwen3-VL 8B and fine-tuned on a robot-specific VQA dataset covering spatial relationships, subtask inference, and low-level action grounding.', 'Cognition tokens are learnable query tokens that attend to visual and linguistic contexts to extract action-relevant representations from intermediate VLM layers.', 'Motion awareness is achieved through a Space-Time Self-Similarity (STSS) encoder integrated after the 9th layer of the vision encoder to capture temporal dynamics.', 'Multi-frame observations are compressed into a single context token after early LLM layers to efficiently capture temporal context while reducing computational complexity.', 'Long-term memory is implemented via a memory queue storing cached cognition features at intervals of H+1 timesteps, processed through a lightweight Transformer with causal attention.', 'The memory module uses a queue size of n_mem=3 and feeds both memory features and original cognition tokens to the action model.', 'The action model generates H+1 future actions conditioned on cognition features, memory features, proprioceptive state, and physical sensory signals.', 'The model is implemented as a flow-matching Diffusion Transformer that learns a denoising velocity field over action trajectories.'], 'section_ids': ['sec_3', 'sec_4', 'sec_5', 'sec_36']}

论点验证

已证实 (85%) RLDX-1 combines four key components: a unified neural architecture integrating diverse functional capabilities; a synthetic data generation pipeline that augments rare manipulation scenarios via motion-consistency filtering; a three-stage training procedure bridging internet-scale pre-trained priors with embodiment-specific deployment; and an inference optimization pipeline that enables real-time control through static graph conversion and operator fusion.
The paper demonstrates all four components through detailed sections and experimental validation: architecture (Section 2), synthetic data pipeline (Section 3.3 with ablation in Table 3), three-stage training (Section 4), and inference optimization (
已证实 (85%) We focus on three such capabilities, including motion awareness, long-term memory, and physical sensing, and address each with a tailored architectural module built on top of a standard flow-matching VLA architecture
The three capabilities (motion awareness, long-term memory, physical sensing) are each addressed with specific architectural modules described in Sections 2.1-2.2. The functional capability experiments (Sections 6.3-6.4) validate each module's effect
已证实 (80%) we propose the Multi-Stream Action Transformer (MSAT), an extension of the Multi-Modal Diffusion Transformer (MM-DiT; Esser et al. 2024; Labs 2024) to action modeling. MSAT assigns a dedicated stream to each modality and couples them through joint self-attention, allowing each modality to retain its own representation while still contributing to action generation.
MSAT is described in detail in p_28-31 with clear architectural specification. The design is demonstrated to work through the main experiments. However, there's no direct ablation comparing MSAT to alternative architectures (e.g., single-stream or si
已证实 (90%) on catching fast-moving objects on conveyor-belt manipulation, RLDX-1 reaches a success rate of over 87.5% while π 0.5 remains below 29.2%
Quantitative results are provided in p_84 and Figure 16. RLDX-1 achieves 100% on seen speeds (S1, S4) and 75% on unseen speeds (S2, S3), averaging to 87.5%. π 0.5 achieves 29.2% average. The numbers are explicitly reported in the functional capabilit
已证实 (90%) the proposed synthetic data results in, e.g., improving success rate by 9.1% on GR-1 Tabletop over training on real data alone
Table 3 in p_97 provides clear quantitative evidence: success rate rises from 41.0% with real data alone to 50.1% at full synthetic scale, a 9.1 percentage point improvement. The ablation controls for synthetic data proportion while keeping other fac
证据不足 (40%) we develop a three-stage training pipeline that progressively specializes the policy from a generalist backbone to a task-specialist deployment model.
While the three-stage training pipeline is described in Section 4 and shown to work, there is no ablation comparing it to alternative training strategies (e.g., two-stage or end-to-end training). The design choice is explained but not empirically jus
已证实 (95%) Under PyTorch Eager (Paszke et al., 2019), the resulting per-step latency reaches 71.2 ms for RLDX-1 on an NVIDIA RTX 5090. [...] Together, the two stages reduce the per-step latency of the all-modality RLDX-1 to 43.7 ms, achieving a 1.63× speedup.
Clear quantitative measurements are provided: p_10 states 71.2 ms under PyTorch Eager, and Table 4 in p_97 confirms 43.7 ms for all-modality model after optimization, with 1.63× speedup explicitly calculated and verified.
已证实 (90%) on GR-1 Tabletop, RLDX-1 achieves 58.7%, outperforming GR00T N1.6, which achieves 47.6%, demonstrating particularly strong performance in humanoid manipulation tasks.
Table 1 in p_66 provides explicit quantitative results: RLDX-1 achieves 58.7% on GR-1 Tabletop, outperforming GR00T N1.6 at 47.6%. The comparison is fair as both are evaluated under the same benchmark protocol.
已证实 (90%) RLDX-1 substantially outperforms π 0.5 in Unseen Object (37.5% to 54.2%) and Unseen Task (45.8% to 54.2%) in versatile intelligence tasks.
Figure 14 in p_76 provides explicit quantitative results for OpenArm humanoid benchmark. RLDX-1 achieves 54.2% on both Unseen Object and Unseen Task, compared to π 0.5's 37.5% and 45.8% respectively.
已证实 (90%) on the ALLEX Object-in-Box Selection task, which requires long-term memory, both GR00T N1.6 and π 0.5 achieve success rates in the 30% range, whereas RLDX-1 achieves a substantially higher success rate of 91.7%.
Figure 16 and p_84 provide explicit quantitative results: RLDX-1 achieves 91.7% on Object-in-Box Selection, while GR00T N1.6 achieves 29.2% and π 0.5 achieves 33.3% (both in the 30% range as claimed).
已证实 (80%) we introduce cognition tokens q, learnable query tokens that are appended to the input token sequence.
Cognition tokens are clearly described as a contribution in p_20 with formal notation. The VLM layer ablation in Table 2a indirectly validates the feature extraction approach, though there's no direct ablation of cognition tokens vs alternatives.
证据不足 (35%) we integrate the module after the 9th layer of the vision encoder (out of 27 layers), motivated by the observation that physically relevant cues are richly represented at around 30% depth
The design choice of placing the module after the 9th layer is justified only by citing external work (Joseph et al., 2026). There is no ablation within this paper comparing different layer positions for the STSS module.
证据不足 (40%) we apply the compression after the 4th layer, rather than after the 2nd layer as in Jang et al. (2025a), to use the DeepStack design of Qwen3-VL (Bai et al., 2025) without compression, where multi-level vision encoder features are fused into the first 4 LLM layers.
The compression after 4th layer is explained as necessary for Qwen3-VL's DeepStack design, but there is no empirical validation or ablation comparing this to the original 2nd layer approach or other alternatives.
证据不足 (30%) In practice, we use a lightweight Transformer module and a memory queue of size n mem = 3.
The memory queue size of 3 is stated without any justification or ablation. No comparison to other queue sizes (e.g., 2, 5, 10) is provided to validate this specific choice.
证据不足 (35%) we apply rotary positional embeddings (RoPE; Su et al. 2024) to the action (A) stream to capture the relative temporal structure within the action chunk better.
RoPE application to the action stream is explained as capturing temporal structure, but there is no ablation comparing RoPE to alternative positional encoding schemes or no positional encoding.
证据不足 (35%) we inject the flow-matching timestep τ as an in-context token rather than through feature-wise modulation (e.g., adaLN; Peebles and Xie 2023). Specifically, τ is encoded via sinusoidal embedding and an MLP and prepended to the A sequence as a single token that participates in attention like any other A tokens
Injecting timestep as in-context token is explained as allowing signal propagation through attention, but there is no ablation comparing this to adaLN or other modulation approaches.
证据不足 (40%) For training efficiency, we resize each image so that each frame yields at most 64 vision tokens while preserving its original aspect ratio.
The 64 vision token limit is justified by 'training efficiency' but there is no analysis of the quality-efficiency trade-off or comparison to other token budgets.
证据不足 (35%) To facilitate adaptation to unseen embodiments, we additionally maintain an embodiment-agnostic projection layer, applied to a small fraction of samples in each batch regardless of source embodiment, providing a strong initialization for downstream fine-tuning.
The embodiment-agnostic projection layer is explained as facilitating adaptation to unseen embodiments, but this benefit is not demonstrated through experiments on actually unseen embodiments or ablation comparing with/without this component.
证据不足 (40%) For ALLEX, we combine in-house teleoperated episodes with 72K synthetic episodes, sampled at a 5:5 ratio during training to balance the two sources given the scale imbalance. For FR3, we combine the 92K episodes from DROID (Khazatsky et al., 2024) with in-house teleoperated episodes collected with the target modalities, sampled at an 8:2 ratio.
The 5:5 and 8:2 sampling ratios are explained as balancing sources, but there is no ablation comparing different ratios to validate these specific choices.
证据不足 (30%) We use chunk horizons of 40 for ALLEX and 16 for FR3, and the memory module covers temporal windows of 120 and 48 past timesteps, respectively.
Chunk horizons of 40 for ALLEX and 16 for FR3 are stated without any justification or ablation comparing different horizon lengths.

... 共 47 个论点

可复现性评估

较低可复现性 (0%)

缺失的复现细节

局限性（作者自述）

论文中未明确列出局限性。

本分析由 PDF 阅读助手 自动生成，仅供参考，不构成学术评审意见。验证结论和可复现性评估基于论文文本自动分析，可能存在偏差。原始论文请参阅 arXiv。

分析时间：2026-05-07T13:00:03+00:00 · 数据来源：Paper Collector