# VeriHarness V0.2 Context Snapshot

Snapshot date: 2026-07-13  
Use: compact starting context for the V0.2 discussion.

## Research Objective

The long-term objective is to establish the importance of multimodal verification inside agentic
work, understand when and how verification should happen, and eventually internalize useful
verification behavior through data, cold start, and reinforcement learning.

Verification is intentionally broader than final acceptance. It may occur after OCR, visual
localization, extraction, rendering, interaction, local comparison, generation/editing, or at a
goal-completion boundary. Image-to-code is the first concrete task, not the generic capability
boundary.

## Baseline Design That Exists

The previous runnable baseline used a two-by-two experiment:

| Condition | Meaning |
| --- | --- |
| `H0P0` | no harness hard gate and no verification-policy prompt |
| `H0P1` | no hard gate; prompt asks the agent to verify consequential claims |
| `H1P0` | harness hard gate; no prompt requirement for self-verification |
| `H1P1` | both prompt-driven verification and harness hard gate |

All conditions shared tools, model profiles, task input, and runtime. `invoke_model` was available
even in P0 so the P axis represented model policy rather than tool access. H0 used a 500-turn
infrastructure guard. H1 had no semantic turn cap and used six outer repair epochs. Online gate
feedback had no score.

This is an empirical baseline, not an automatically accepted V0.2 design.

## Baseline Evidence Contract

- `invoke_model` calls a fresh isolated model with only explicit instruction, declared inputs,
  model profile, recipe, and output contract.
- Visual locate, inspect/OCR, align, compare, image generation/editing, and custom are general
  capabilities; detailed recipes can specialize instructions.
- Images must be attached through the canonical image input path. Text that merely names an image
  path is not multimodal input.
- Transport success and schema-valid output do not imply useful evidence.
- Irrelevant OCR, guessed text, plan-only answers, missing verdicts, and claim-irrelevant output
  are insufficient.
- Each observation is linked privately to exact input hashes and the artifact version observed.
- Editing the artifact invalidates an old candidate observation for claims affected by the edit.

## Baseline Harness Properties

- standalone Claude Agent SDK runtime;
- general model router plus versioned recipes;
- explicit `invoke_model` and `propose_finish` tools;
- persistent agent session and outer repair controller;
- tri-state H1 result: accept, reopen on grounded mismatch, or blocked on required infrastructure;
- fail-closed retry and terminal infrastructure failure;
- separate agent model calls and private gate model calls;
- transcript separated from append-only structured event trace;
- stable claim/object identity and immutable per-attempt claim-bank audits;
- task-specific HTML/DOM/rendering/gate logic outside the generic core;
- posthoc offline evaluator that does not feed the agent or control epochs.

## Baseline Offline Outcome Measure

The implemented image-to-code evaluator used four weighted dimensions:

| Dimension | Weight | Purpose |
| --- | ---: | --- |
| Canvas | 10% | render existence and source/candidate canvas size |
| Text Geometry | 40% | source text grounding against live candidate DOM geometry |
| Fresh Visual Bank | 30% | frozen source-derived visual questions shared by conditions |
| Structure Audit | 20% | live text, layout structure, and anti-shortcut properties |

The result was a weighted 0-to-5 score with a strict pass line at 4.0. The evaluator ran only
after all agent activity stopped.

## Frozen Case1 Evidence

| Condition | Total | Canvas | Text Geometry | Fresh Visual | Structure |
| --- | ---: | ---: | ---: | ---: | ---: |
| `H0P0` | 2.44 | 5.00 | 0.00 | 4.38 | 3.12 |
| `H0P1` | 2.62 | 5.00 | 0.00 | 5.00 | 3.12 |
| `H1P0` | 4.08 | 5.00 | 3.64 | 5.00 | 3.12 |
| `H1P1` | 3.26 | 5.00 | 1.59 | 5.00 | 3.12 |

Allowed interpretation: harness-only produced positive evidence on this single artifact.

Disallowed interpretation: verification, prompt-driven multimodal verification, or the combined
condition has been generally proven to improve final quality.

## Failures And Caveats Already Learned

1. Historical P1 calls did not receive image bytes because the router treated `kind:image` as
   text. The code was repaired and smoke-tested, but the frozen P1 outcomes remain invalid for
   tool-driven multimodal verification.
2. The private H1 gate used a separate working image transport, but its semantic judgments could
   still be inconsistent or ambiguously prompted.
3. Greedy one-to-one Text Geometry matching can convert punctuation, whitespace, line-break,
   Unicode, or repeated-text binding failures into apparent missing text.
4. An accumulating gate can expand its acceptance contract across epochs and induce verifier-
   specific overfitting. Historical banks must remain immutable and inspectable.
5. One case cannot establish cross-case robustness, cross-task generality, or a causal headline.

## RL-Relevant Record

The accepted baseline episode unit separates observation quality, agent response, and artifact
effect. Transcript and structured event trace remain separate. This enables later reconstruction
of verification choices and repair effects without injecting bookkeeping into the agent prompt.
No RL training or internalization result exists yet.

## Research Interface Already Available

The project board already demonstrates the baseline with a dedicated summary, four conditions,
four evaluator dimensions, case1 audit, complete English prompts, reader-only Chinese prompt
translations, and a four-condition case1 trajectory viewer.

## Invariants Worth Carrying Into Discussion

- Verification should be studied as behavior across a workflow.
- Evidence admission begins with real input integrity.
- Prompt policy cannot enforce transport, isolation, schema, or failure semantics by itself.
- Online repair and offline outcome scoring should not be conflated.
- Required verification fails closed.
- Freshness, provenance, transcript, and event trace are first-class research data.
- Core capabilities and task-specific mechanisms must stay separable.

## Mechanisms That Must Be Re-argued

- the H/P two-axis experiment;
- the hard gate as the principal goal-driven mechanism;
- one accumulating claim bank;
- a fixed six-epoch outer loop;
- the current capability and recipe taxonomy;
- which checks are deterministic versus model-judged;
- the four-dimension evaluator and its weights;
- how verification episodes become training data;
- which current code should be copied, redesigned, or discarded.

## Discussion Protocol

Start with the newly proposed major change. Resolve one core design question at a time. Record the
accepted decision and its experimental consequence in the V0.2 decision log before proceeding.
Do not treat this context snapshot as a decision to preserve the old implementation.