TL;DR
HumanNet introduces a one-million-hour human-centric video corpus showing that 1,000 hours of egocentric pretraining matches or surpasses 100 hours of real-robot data, substantially closing the gap to a 20,000-hour robot baseline.

论文原始摘要

Progress in embodied intelligence increasingly depends on scalable data infrastructure. While vision and language have scaled with internet corpora, learning physical interaction remains constrained by the lack of large, diverse, and richly annotated human activity data. We present HumanNet, a one-million-hour human-centric video corpus that captures how humans interact with the physical world at scale. HumanNet spans both first-person and third-person perspectives and covers fine-grained activities, human-object interactions, tool use, and long-horizon behaviors across diverse real-world environments. Beyond raw video, the dataset provides interaction-centric annotations, including captions, motion descriptions, and hand and body-related signals, enabling motion-aware and interaction-aware learning. Beyond scale, HumanNet introduces a systematic data curation paradigm for embodied learning, where human-centric filtering, temporal structuring, viewpoint diversity, and annotation enrichment are treated as first-class design principles. This design transforms unstructured internet video into a scalable substrate for representation learning, activity understanding, motion generation, and human-to-robot transfer. We conduct a first-step validation on the value of this design through controlled vision-language-action ablation: under a fixed set of validation data, continued training from the Qwen VLM model with 1000 hours of egocentric video drawn from HumanNet surpasses the continued training with 100 hours of real-robot data from Magic Cobot, indicating that egocentric human video could be a scalable and cost-effective substitute for robot data. By building this project, we aim to explore the opportunity to scale embodied foundation models using human-centric videos, rather than relying solely on robot-specific data.

Paper Collector 中文速览

百万小时人本视频数据集，推动具身智能规模化

方法概述

收集百万小时第一和第三视角视频，涵盖细粒度活动、人机交互、工具使用等。提供交互中心化标注，包括字幕、运动描述、手部和身体信号。通过受控的视觉-语言-动作消融实验，验证从HumanNet抽取1000小时视频训练优于100小时真实机器人数据

核心贡献

构建百万小时人本视频数据集，提出系统性数据整理范式，验证人本视频可替代机器人数据用于具身智能训练

原始来源与核验范围

核验仅覆盖论文正文中的主张、证据片段和参考文献关系。NGJOO 未独立运行作者代码、重做实验或验证真实部署效果；页面中的可复现性分数是文档完整度评估，不是复现实验结果。

27
已证实
5
证据不足
5
无法验证
N/A
可复现性
置信度
73%

核心问题

Can large-scale human-centric video serve as a scalable data source for embodied learning systems, and to what extent can egocentric human video substitute for real robot training data?

核心方法

The authors constructed HumanNet through a three-stage pipeline: data collection via keyword discovery and crawling, data processing with deduplication and quality filtering, and annotation including 3D pose detection, motion retargeting, and LLM-assisted captioning. Validation used controlled post-training comparison under the LingBot-VLA architecture across four configurations with varying pretraining sources.

方法组件

论点验证

已证实 (80%) We introduce HumanNet, a one-million-hour human-centric video corpus spanning first-person and third-person views of fine-grained physical activities, organized by a multi-axis taxonomy over source type, viewpoint, task structure, environment, interaction style, motion category, and metadata availability.
已证实 (85%) We describe a full curation pipeline covering acquisition, human-centric filtering, viewpoint characterization, segmentation, deduplication, quality control, privacy review, and caption and motion annotation, turning heterogeneous web video into infrastructure for representation learning, motion-aware video modeling, and embodied pretraining.
证据不足 (60%) We empirically validate the corpus through a controlled vision-language-action post-training study, showing that 1,000 hours of egocentric pretraining from HumanNet matches or modestly surpasses 100 hours of real-robot from Magic Cobot pretraining under an identical downstream regime, and substantially closes the gap to a 20,000-hour real-robot baseline.
已证实 (75%) HumanNet therefore treats large-scale human-centric video as the primary data source: first-person recordings capture actor-centered intent and hand-object contact, while third-person recordings capture full-body motion, spatial context, multi-person interaction, and the geometry of activity in the surrounding scene.
已证实 (70%) We define human-centric video as footage in which human activity is the organizing signal of the clip. A clip may be first-person or third-person, but it must contain physically meaningful behavior such as manipulating objects, using tools, navigating through task-relevant space, assembling or disassembling items, operating appliances or interfaces, transporting objects, coordinating with other people, or executing multi-step procedures with visible state changes in the environment.
The definition is clearly stated and operationally specified. Paragraph 15 defines human-centric video with concrete examples of qualifying behaviors (manipulating objects, tool use, navigation, etc.) and exclusions (passive or weakly grounded video)
已证实 (75%) Viewpoint diversity means that first-person and third-person sources are both retained and explicitly indexed, allowing models to learn complementary actor-centered and observer-centered cues.
已证实 (70%) Physical relevance means that the data should preserve cues useful for embodied learning, including hand-object proximity, full-body motion, state changes, action ordering, procedural structure, and scene context.
已证实 (75%) Pretraining readiness means that the dataset must be organized so it can support modern large-scale training pipelines, including chunking, metadata indexing, quality filtering, caption labels, motion annotations, and optional alignment with text or structured labels.
The design choice is described with specific organizational elements. Paragraph 16 lists chunking, metadata indexing, quality filtering, caption labels, motion annotations. The pipeline (paragraphs 22-26) implements these components.
已证实 (70%) Rather than treating this heterogeneity as noise, we index the corpus through a small set of factors that determine its value for human-centric video learning: where the data comes from, which viewpoint it uses, what kind of physical activity it contains, and what supervision signals are available after processing.
The indexing factors are clearly described in paragraph 19: source type, viewpoint, physical activity content, and supervision signals. This is a methodological design choice that is fully specified.
已证实 (70%) Interaction content is organized around physically grounded behavior rather than a closed set of semantic labels. The main emphasis is on manipulation, tool use, object transport, locomotion, full-body movement, environment state changes, multi-person coordination, and long-horizon procedures that combine motion with human-object or human-scene interaction.
The interaction content organization is described in paragraph 20 with specific behavior categories. The multi-label approach is mentioned. This is a conceptual design choice that is well-articulated.
已证实 (80%) The end-to-end construction pipeline is organized into three stages: data collection, data processing, and annotation. This staged design cleanly separates source acquisition from clip-level cleaning and from supervision generation, so that each stage can be audited, extended, or rerun independently as the corpus scales toward one-million-hour coverage.
The three-stage pipeline is clearly described and visualized. Paragraph 22 states the three stages (collection, processing, annotation) and explains the separation benefits. Figure 3 is referenced. Each stage is detailed in subsequent paragraphs.
已证实 (75%) The collection stage couples keyword discovery with content search and retrieval. A small set of seed keywords is iteratively enlarged through keyword expansion, keyword-based crawling and cleaning, channel-level crawling, and integration of existing data sources, producing a unified keyword repository that drives subsequent retrieval.
The collection stage methodology is comprehensively described in paragraph 23, including keyword discovery, expansion, crawling approaches, and source integration. This is a methodological description that is fully specified.
已证实 (80%) The processing stage converts raw videos into clip-level training samples and applies all quality control needed for downstream use. Each video is passed through de-duplication and normalization to remove near-identical copies and to unify frame rate, resolution, and container format; content filtering to retain clips with meaningful human action and observable motion; quality filtering to discard recordings with severe motion blur, heavy occlusion, static framing, or other defects that undermine learning; scene splitting that segments long videos at visual changes so that unrelated activities are not merged into a single sample; and finally video clipping that produces fixed-granularity segments.
已证实 (75%) The annotation stage enriches the processed clips with both geometric and semantic supervision. 3D hand and body pose detection recovers fine-grained motion structure; monocular SLAM estimates camera trajectory for first-person clips that satisfy stability and parallax requirements; and a retargeting module aligns recovered human motion with a unified humanoid skeleton, designating clips as robot-ready when the retargeting error remains below 15 mm and valid-frame coverage exceeds 60%.
已证实 (70%) An LLM-assisted captioning module produces video captions, motion descriptions, and activity classifications, which are normalized against any narrations or metadata inherited from the source.
已证实 (70%) The action vocabulary is dominated by physically grounded manipulation verbs acting on recurring everyday objects, consistent with the design intent that the corpus emphasizes contact-rich, transformation-inducing behavior rather than passive observation.
已证实 (70%) The scene hierarchy spreads clips across a wide range of indoor and outdoor environments instead of concentrating on a single domain, and the activity-category distribution exhibits a pronounced long tail.
The finding is referenced to Figure 2 in paragraph 28. The paper describes the scene hierarchy spreading across indoor/outdoor environments and the long-tail activity-category distribution. The figure is cited as evidence.
已证实 (65%) The motion-score and motion-length distributions are both heavy-tailed yet well bounded by their statistics, reflecting a corpus dominated by short, focused interaction units while still retaining longer and more vigorous segments needed for temporal context and procedural learning.
已证实 (65%) The per-category breakdown further makes the heterogeneity of the corpus explicit, with athletic and outdoor families showing longer, higher-magnitude motion, while daily activities and game-character actions concentrate on shorter, finer-grained segments.
The finding is referenced to Figure 5 in paragraph 28. The paper describes per-category breakdown showing athletic/outdoor families with longer, higher-magnitude motion versus daily activities with shorter, finer-grained segments. Without the figure,
证据不足 (50%) We find that 1,000 hours of egocentric video drawn from HumanNet attains validation loss on par with, and on several task groups below, that of a model initialized from 100 hours of real-robot data.

... 共 37 个论点

可复现性评估

较低可复现性 (0%)

缺失的复现细节

局限与证据边界

本分析由 PDF 阅读助手 自动生成，仅供参考，不构成学术评审意见。验证结论和可复现性评估基于论文文本自动分析，可能存在偏差。原始论文请参阅 arXiv。

分析时间：2026-05-18T13:19:15+00:00 · 数据来源：Paper Collector