ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration

核心问题

How can autonomous research systems ensure research integrity when long-horizon tasks performed by single agents are unreliable? The paper investigates whether treating assurance as a first-class workflow layer with cross-family executor/reviewer separation can address the failure mode of plausible unsupported success.

核心方法

{'approach': 'ARIS implements a three-layer architecture mapping to three bottlenecks: persistent research state via a research wiki, modular execution via 65+ Markdown skill files, and independent assurance via cross-family executor/reviewer pairings. The core mechanism is a critique-to-action loop where reviewers from different model families score artifacts and return action items until convergence, complemented by a three-stage evidence-to-claim audit cascade for experimental integrity.', 'key_components': ['The critique-to-action loop involves an executor producing an artifact and a cross-family reviewer providing scores and action items until convergence or maximum rounds.', 'Reviewer independence is maintained by having reviewers read referenced artifacts directly rather than relying on executor summaries.', 'Reviewers are configured along two axes: access scope (document-only, artifact-augmented, repository-level) and context policy (fresh versus cross-round).', 'Automatic debugging assigns failures to predefined error classes with class-specific remediation and up to three retry attempts.', 'An independently configured third model can provide diagnosis through a dedicated rescue step when remediation attempts fail.', 'The assurance stack addresses the risk that executor agents may use deceptive methods to improve peer review scores during dialogue.', 'The stack includes a three-stage evidence-to-claim audit cascade for experimental integrity.', 'A manuscript assurance layer provides checks for prose, proof, and presentation quality.', 'System-wide controls include effort levels and reviewer routing to set audit depth and reviewer backend.', 'The assurance stack operationalizes bottleneck (iii) of independent assurance from Section 1.'], 'section_ids': ['sec_4', 'sec_5']}

论点验证

... 共 48 个论点

可复现性评估

较低可复现性 (0%)

局限性（作者自述）

• This is a single trajectory on one paper; we do not generalize from it. This run should be read as evidence that the harness can operationalize claim pruning and review-driven revision in one realistic trajectory, not as causal evidence that cross-family review is superior to same-family review.
• Aris cannot guarantee that any output is correct, novel, or scientifically sound. LLM outputs can include factual hallucinations and methodological gaps; cross-model review reduces some failure modes without eliminating them.
• The three-stage audit cascade can catch common integrity failures, but it cannot detect every error, inconsistency, or fabrication. It is an advisory safety net, not a formal verification system.
• The review loop can amplify reviewer biases: if the reviewer consistently demands a particular methodology, the loop may overfit to the reviewer model's preferences rather than improve broader scientific quality. Over-iteration past diminishing returns can degrade paper quality.
• Repository-level review may send source code to external LLM APIs, raising confidentiality concerns. Users should not enable repository-level review on repositories containing sensitive code or secrets unless an approved local-only review path is available.