# PaperFit 2026: Vision-in-the-Loop Typesetting Optimization

Status: captured-for-framing  
Bucket: visual-feedback / document-typesetting  
Added: 2026-06-24  

## Citation

Bihui Yu, Xinglong Xu, Junjie Jiang, Jiabei Cheng, Caijun Jia, Siyuan Li, Conghui He, Jingxuan Wei, and Cheng Tan. "PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents." arXiv:2605.10341, 2026.

- Paper: https://arxiv.org/abs/2605.10341
- HTML: https://arxiv.org/html/2605.10341v1
- Repository: https://github.com/OpenRaiser/PaperFit

## One-line Takeaway

PaperFit is a concrete application of our broader framing: multimodal verification lets an agent continue working inside an existing source project by rendering the artifact, inspecting visual failures, applying constrained source edits, and validating the result after every edit.

## Why It Matters Here

PaperFit should not be read as only a LaTeX formatting tool. For our survey, it is important because it has the exact loop shape we care about:

```text
existing LaTeX project
  -> compile and render PDF pages
  -> inspect visual/layout defects
  -> produce source-level repair
  -> recompile and re-render
  -> gate on visual quality, page budget, and content integrity
```

This is not blank-slate generation and not simple "replication." It is outcome-conditioned continuation: the agent starts from an existing artifact and uses rendered evidence to decide what code/source change should happen next.

## Problem

The paper formalizes Visual Typesetting Optimization (VTO): transforming a compilable LaTeX manuscript into a visually polished and page-budget-compliant PDF.

The key observation is that successful compilation is insufficient. A PDF can still have:

- misplaced floats;
- overflowing equations or tables;
- inconsistent table scaling;
- widows, orphans, and poor page balance;
- excessive whitespace or page-budget violations;
- template migration issues.

Text-only and log-only tools miss many of these because typesetting quality is two-dimensional and visual. A source edit can also have non-local cascading effects across pages, so verification has to happen after edits, not only before.

## Method Shape

PaperFit's workflow can be summarized as a sense-act-verify loop:

| Stage | Evidence / Action |
| --- | --- |
| Sense | Fuse LaTeX source, logs, PDF metadata, and rendered page images |
| Diagnose | Build structured defect records using a typesetting defect taxonomy |
| Act | Apply constrained source-level repairs while preserving academic content |
| Verify | Recompile, re-render, reinspect, and gate the output after each edit |

The paper emphasizes that visual feedback alone is not enough. Its structured diagnosis, constrained repair policy, content protection, and gated validation are part of what makes the loop reliable.

## Connection to Our Project

PaperFit is a strong motivating example for multimodal verification in agentic coding loops:

- It shows why execution success is not equal to artifact success.
- It uses rendered pages as evidence, not as decoration.
- It turns visual failures into source-level edits.
- It checks for cascading effects by validating after each edit.
- It distinguishes legitimate repairs from pseudo-fixes.

This maps cleanly to our candidate task form:

```text
existing system + intent/constraints + rendered observation
  -> multimodal verification
  -> actionable repair feedback
  -> source/code revision
```

## Difference From Our Intended Contribution

| PaperFit | Our broader project |
| --- | --- |
| Specific domain: LaTeX scientific document typesetting | General loop-level multimodal verification for coding agents |
| Output artifact: PDF pages | UI, chart, document, dashboard, diagram, slides, and interaction traces |
| Verifier target: page layout and publication constraints | Intent satisfaction, visual/spatial/semantic correctness, regressions, and behavior |
| Main contribution: VTO system and benchmark | Problem framing, task taxonomy, verifier abstraction, and prototype evidence |

## Design Ideas To Borrow

- Treat rendered artifacts as first-class evidence.
- Preserve structured defect records rather than only free-form feedback.
- Separate permitted repairs from pseudo-fixes.
- Protect user/scientific content during automated repair.
- Re-run verification after every edit because changes can be non-local.
- Compare against naive visual multi-round agents to show that "has images" is not enough.

## Evidence Strength

| Claim | Evidence | Strength | Relevance |
| --- | --- | --- | --- |
| Compilable source can still fail the user-facing artifact goal. | PaperFit's task definition and defect categories focus on visual PDF layout issues after successful compilation. | strong | high |
| Visual verification needs to be in the loop, not only at the end. | The system re-renders and validates after source edits to catch cascading layout effects. | strong | high |
| Multimodal verification benefits from structure and constraints. | PaperFit reports a gap between naive visual multi-round repair and the full structured workflow. | medium-to-strong | high |

## Open Questions for Our Survey

- How do we generalize PaperFit's content-protection idea to UI, dashboards, charts, and interaction traces?
- Which PaperFit-style gates map to our minimal prototype: compile/render success, visual pass/fail, content integrity, regression checks?
- Can we design a small task where naive visual feedback is not enough, so structured verifier output matters?
- Should document/PDF optimization be one of our first benchmark families alongside image-to-code?