NeurIPS 2026 · Vision-language reasoning

Reason wider.
Stay grounded.

Saliency-Aware Principle Selection scales inference over compact reasoning behaviors—not one ever-longer chain—and lets visual tools advise selection without writing the answer.

01Training-freeNo parameter updates
02Tool-advisoryTools select, never answer
03Route-parallelWider without a longer serial chain
visual support
image reason reason reason guess
one long chainSAP
principle 01inspect → verify → answer
principle 02localize → compare → answer
principle 03test alternatives → answer
01 / THE PROBLEM

More tokens do not necessarily mean more vision.

A vision-language model sees the image at the start, but regenerates text at every step. As the chain grows, an early visual mistake can be repeated until it reads like evidence.

Directly inserting tool output is not a clean fix: one incorrect detection can become a premise for everything that follows. The real question is how an agent can use tools while remaining able to disagree with them.

01Image encoded onceVisual evidence enters at the beginning.
02Text accumulatesThe trajectory becomes self-referential.
03Error hardensAn unsupported object survives into the answer.
02 / THE SHIFT

Search over how to reason, not just what token comes next.

SAP evolves short principles. Each principle produces independent routes from the original image and prompt. A frozen grounding tool checks object support; agreement, diversity, and uncertainty help decide which behaviors survive.

1

Propose

A population of compact reasoning principles.

2

Route

Independent, parallel reasoning from the original input.

3

Ground

Check cited objects without letting tools write the answer.

4

Evolve

Retain stronger principles and form the next generation.

Core constraintThe tool supplies selection evidence. The VLM keeps control of reasoning and answering.
03 / RESULTS

Grounding improves without giving up broad capability.

All hallucination comparisons below use the same Qwen3-VL-8B stack, benchmark items, evaluator interfaces, and decoding budget. SAP values are averaged over three seeds.

Object hallucination

Higher visual fidelity across three evaluations

Baselines SAP
Bars, left to right: Instruct, MFCD, M3ID, ICD, MaskCD, SAP. Higher is better.
16-benchmark suite

Average performance, excluding MME

+1.2 vs. Instruct
Reasoning, knowledge, OCR, documents, diagrams, and real-world imagery under matched backbone and inference-time decoding budget.
Controlled experiments

What makes the gain hold?

The ablations isolate the effect of principles, evolutionary selection, and transfer beyond the default backbone.

A / DEPTH

Principles matter more as the chain grows.

The POPE gap is largest at full length, where an unconstrained chain drifts most.

L/8L/4L/2L
with principles without
B / MODULES

Routes and evolution are complementary.

Removing either component loses the full configuration's average advantage.

Visual scale: 70–77; exact values shown.
C / TRANSFER

The mechanism is not tied to one model.

SAP transfers across architectures and helps every tested Qwen3-VL scale.

InternVL3.5 · MMMU

74.675.3

DeepSeek-VL2 · MMMU

51.256.7

Qwen3-VL · 2B

60.466.6

Qwen3-VL · 30B

77.378.1

base+ SAP
04 / PAPER

Saliency-Aware Multi-Route Thinking

Mingjia Shi · Yinhan He · Yaochen Zhu · Cassie Dong · Jundong Li

University of Virginia · Meta

PDF Source