Propose
A population of compact reasoning principles.
Saliency-Aware Principle Selection scales inference over compact reasoning behaviors—not one ever-longer chain—and lets visual tools advise selection without writing the answer.
A vision-language model sees the image at the start, but regenerates text at every step. As the chain grows, an early visual mistake can be repeated until it reads like evidence.
Directly inserting tool output is not a clean fix: one incorrect detection can become a premise for everything that follows. The real question is how an agent can use tools while remaining able to disagree with them.
SAP evolves short principles. Each principle produces independent routes from the original image and prompt. A frozen grounding tool checks object support; agreement, diversity, and uncertainty help decide which behaviors survive.
A population of compact reasoning principles.
Independent, parallel reasoning from the original input.
Check cited objects without letting tools write the answer.
Retain stronger principles and form the next generation.
All hallucination comparisons below use the same Qwen3-VL-8B stack, benchmark items, evaluator interfaces, and decoding budget. SAP values are averaged over three seeds.
The ablations isolate the effect of principles, evolutionary selection, and transfer beyond the default backbone.
The POPE gap is largest at full length, where an unconstrained chain drifts most.
Removing either component loses the full configuration's average advantage.
Visual scale: 70–77; exact values shown.SAP transfers across architectures and helps every tested Qwen3-VL scale.
74.675.3
51.256.7
60.466.6
77.378.1
Mingjia Shi · Yinhan He · Yaochen Zhu · Cassie Dong · Jundong Li
University of Virginia · Meta