IndusAgent: Reinforcing Open-Vocabulary Industrial Anomaly Detection with Agentic Tools - AI 论文深度分析

TL;DR
IndusAgent enhances MLLMs for industrial anomaly detection through supervised fine-tuning, four specialized tools, and agentic reinforcement learning with accuracy-gated rewards. It achieves 83.4% accuracy across five benchmarks, outperforming SOTA by 9.

论文原始摘要

Multimodal large language models (MLLMs) have shown remarkable capability in bridging visual perception and textual reasoning, enabling zero-shot understanding across diverse industrial scenarios. However, their performance in open-vocabulary industrial anomaly detection (IAD) is often limited by domain-misaligned reasoning and hallucinated structural inferences. To address these challenges, we propose \textbf{IndusAgent}, a tool-augmented agentic framework for open-vocabulary IAD. Specifically, we first construct \textbf{Indus-CoT}, a structured dataset that integrates global visual observations, high-resolution local patches, and expert normalcy priors, providing supervision for fine-tuning the model on rigorous industrial inspection trajectories. Building on this, IndusAgent dynamically orchestrates a set of external tools, including dynamic region cropping, high-frequency feature enhancement, and prior retrieval, thus enabling the agent to actively resolve visual ambiguities and disentangle subtle anomalies. Furthermore, we introduce a gated reinforcement learning objective that jointly optimizes anomaly classification, localization accuracy, anomaly type reasoning, and efficient tool usage, ensuring that tool invocation occurs only when beneficial. Extensive evaluations on five industrial anomaly benchmarks, including MVTec-AD, VisA, MPDD, DTD, and SDD, demonstrate that IndusAgent achieves state-of-the-art zero-shot performance among all existing methods, validating our robustness and generalization capacity.

Paper Collector 中文速览

工具增强智能体框架，提升开放词汇工业异常检测性能

方法概述

构建融合全局视觉、局部高分辨率块和专家先验的Indus-CoT数据集；设计智能体动态调度区域裁剪、高频特征增强和先验检索等外部工具；引入门控强化学习联合优化异常分类、定位、类型推理和工具使用效率。

核心贡献

提出IndusAgent框架，构建Indus-CoT数据集，设计动态工具调度机制和门控强化学习目标，实现SOTA零样本工业异常检测。

原始来源与核验范围

核验仅覆盖论文正文中的主张、证据片段和参考文献关系。NGJOO 未独立运行作者代码、重做实验或验证真实部署效果；页面中的可复现性分数是文档完整度评估，不是复现实验结果。

已证实

证据不足

无法验证

N/A

可复现性

置信度

87%

核心问题

How can Multimodal Large Language Models be enhanced to overcome domain-misaligned reasoning, perceptual dilution, and poor generalization in open-vocabulary industrial anomaly detection?

核心方法

The framework combines three stages: construction of Indus-CoT dataset (~3,000 tool-integrated reasoning trajectories), supervised fine-tuning of Qwen3-VL-8B, and tool-augmented reinforcement learning using Group Relative Policy Optimization with an accuracy-gated reward mechanism. Four specialized tools (crop, prior, enhance, measure) address specific IAD failure modes through multi-step autoregressive reasoning.

方法组件

The framework consists of three stages: dataset construction, supervised fine-tuning, and tool-augmented reinforcement learning.
Indus-CoT bridges visual perception and tool execution through synthesized image-query trajectories.
SFT aligns the VLM with structured industrial diagnostic trajectories and tool-use syntax.
Tool-Augmented RL uses a hierarchical reward balancing tool-usage correctness, anomaly interpretation, and reasoning coherence.

论点验证

已证实 (95%) we propose IndusAgent, a unified framework that synergizes domain-specific reasoning with autonomous tool orchestration

已证实 (85%) We introduce a unified paradigm that integrates autonomous, multiround tool orchestration with MLLMs for industrial anomaly detection, effectively transcending the resolution and semantic limitations inherent in passive visual perception

已证实 (90%) We construct Indus-CoT, a structured reasoning dataset that encodes industrial inspection trajectories with global observations, localized evidence, normalcy priors, and final defect judgments

已证实 (90%) We formulate a cascading Agentic RL objective that utilizes a multiplicative gate to seamlessly integrate tool utility with diagnostic task efficacy

已证实 (80%) IndusAgent achieves state-of-the-art results across five challenging benchmarks (MVTec-AD, VisA, DTD, MPDD, and SDD), especially outperforming SOTA method by 9.3% on MVTec

已证实 (95%) we instantiate the VLM as an agentic policy π θ based on Qwen3-VL-8B
The paper explicitly states this design choice in p_8: 'we instantiate the VLM as an agentic policy π_θ based on Qwen3-VL-8B [4]'. This is a straightforward, verifiable design decision.

已证实 (95%) The policy interacts with a customized tool space T = {T crop , T prior , T enhance , T measure }
The paper explicitly defines the tool space T = {T_crop, T_prior, T_enhance, T_measure} in p_8 and describes each tool in detail in p_11-p_12.

已证实 (90%) We instantiate four tools to address typical IAD failure modes. T crop extracts high-resolution patches from suspicious regions to recover fine-grained defects diluted by global encoding. T prior retrieves normalcy priors describing defect-free geometry, texture, and structural patterns, providing a comparison anchor for distinguishing true defects from acceptable variations. T enhance applies lightweight image-processing operations, such as contrast enhancement and edge extraction, to highlight low-contrast texture changes. T measure computes geometric relations, such as distances, angles, and relative positions, to verify misalignment, deformation, missing parts, and abnormal spacing.
The paper provides detailed descriptions of all four tools in p_11-p_12 and extended specifications in p_66-p_71, including their purposes, implementations, and how they address specific IAD failure modes.

已证实 (90%) We sample images from Real-IAD and construct about 3,000 reasoning trajectories, with roughly balanced normal and anomalous samples
The paper explicitly states in p_14: 'We sample images from Real-IAD [84] and construct about 3,000 reasoning trajectories, with roughly balanced normal and anomalous samples'. Specific numbers are provided.

已证实 (90%) To prevent category leakage, we remove all Real-IAD categories overlapping with the evaluation benchmarks, including DTD, MPDD, MVTec-AD, SDD, and VisA, using both exact matching and semantic normalization for naming variants
The paper describes the category leakage prevention methodology in detail in p_14 and p_56, including specific categories removed (toothbrush, zipper, pcb, transistor1) and the semantic normalization approach for naming variants.

已证实 (90%) For each query image, no paired normal reference image is provided to the teacher model. The teacher receives only the query image and task instruction, infers the expected defect-free appearance from its internal visual-language knowledge and general industrial priors
The paper explicitly states this design in p_15: 'For each query image, no paired normal reference image is provided to the teacher model. The teacher receives only the query image and task instruction...'

已证实 (85%) To improve data quality, we further apply self-correction and LLM-as-a-judge validation to repair invalid outputs, score candidate trajectories, and retain the highest-quality valid trajectory

已证实 (90%) For T crop , we avoid using ground-truth boxes during execution and instead adopt an unsupervised foreground extraction procedure, combining background estimation, image differencing, Otsu thresholding, morphological operations, and a center-crop fallback

已证实 (90%) we introduce a Supervised Fine-Tuning (SFT) stage to cold-start Qwen3-VL-Instruct (8B) with structured industrial diagnostic trajectories before reinforcement learning
The paper describes the SFT stage in p_18-p_21, including the rationale (cold-start before RL), the base model (Qwen3-VL-Instruct 8B), and the training objective.

已证实 (90%) we implement a selective masking strategy during training. The objective minimizes the negative log-likelihood exclusively over the generated tokens of the reasoning process
The paper describes the selective masking strategy in p_20-p_21 with the mathematical objective that minimizes negative log-likelihood exclusively over generated tokens of the reasoning process.

已证实 (90%) To optimize the agent's decision-making process without the prohibitive memory costs associated with traditional actor-critic architectures, we utilize Group Relative Policy Optimization (GRPO)
The paper describes GRPO in p_22-p_24, including the rationale (avoiding memory costs of actor-critic), the mathematical formulation, and the advantage estimator derivation.

已证实 (90%) We propose an Accuracy-Gated reward that couples tool usage with final diagnostic correctness, so that auxiliary rewards are activated only when the basic anomaly judgment is correct
The paper provides complete mathematical formulation of the Accuracy-Gated reward in p_25-p_27 and p_38-p_55, showing how R_acc serves as a multiplicative gate that activates auxiliary rewards only when the binary judgment is correct.

已证实 (90%) R acc ∈ {0, 1} evaluates whether the final binary anomaly judgment is correct and serves as a multiplicative gate, ensuring that localization, type prediction, and tool-usage rewards are credited only when the final diagnosis is correct
The paper explicitly describes R_acc in p_27 and p_41, stating it evaluates binary anomaly classification correctness and serves as a multiplicative gate.

已证实 (90%) R loc measures the overlap between the predicted anomaly region and the ground-truth region using IoU
The paper describes R_loc in p_27 and provides the mathematical formulation using IoU in p_42-p_43.

已证实 (90%) R type evaluates the correctness of the predicted anomaly type based on its semantic distance to the ground-truth category
The paper describes R_type in p_27 and p_44, explaining it evaluates predicted anomaly type based on semantic distance in a hierarchical anomaly taxonomy.

... 共 42 个论点

可复现性评估

较低可复现性 (0%)

缺失的复现细节

Code is not available (only stated 'will be released')
Data is not available (Real-IAD training set and Indus-CoT dataset)
No specific hyperparameters provided (learning rate, batch size, epochs, optimizer settings)
No random seeds specified for reproducibility
No hardware specifications (GPU type, memory, number of GPUs)
No training time or computational cost details
No specific VLM architecture details or base model specified
No details on reinforcement learning algorithm hyperparameters (reward weights, policy optimization settings)
No details on tool definitions and implementations
No details on Indus-CoT dataset construction prompts and procedures

局限与证据边界

the active inspection process introduces additional inference overhead compared with single-pass MLLM inference, since external tools such as cropping, enhancement, and prior retrieval require extra computation
the framework depends on the reliability of tool feedback; inaccurate crops, noisy enhanced maps, or incomplete normalcy priors may mislead the agent and affect the final diagnosis
our current experiments mainly focus on image-level anomaly judgment, while more fine-grained evaluations, such as pixel-level localization, region-level grounding, and tool-use efficiency analysis, are needed to better understand the agent's diagnostic behavior
Indus-CoT is generated with the assistance of a strong teacher model and rule-based validation, which may introduce teacher or prompt-template bias
Future work will explore more efficient tool-use policies, stronger tool robustness, and more diverse expert supervision for practical industrial deployment

本分析由 PDF 阅读助手自动生成，仅供参考，不构成学术评审意见。验证结论和可复现性评估基于论文文本自动分析，可能存在偏差。原始论文请参阅 arXiv。

分析时间：2026-05-25T07:25:26+00:00 · 数据来源：Paper Collector