# VeriHarness V0.1 Progress Snapshot

Lifecycle: frozen baseline  
Frozen: 2026-07-13  
Purpose: preserve the complete research and implementation state before the V0.2 redesign.

## 1. Research Target

The project studies whether explicit multimodal verification is important inside an agentic
coding loop and whether high-quality verification behavior can eventually be internalized by a
model through cold start and reinforcement learning.

V0.1 deliberately treated verification as broader than a final evaluator call:

- **Tool-driven verification** can happen after OCR, visual localization, extraction, rendering,
  interaction, local comparison, image generation, or another consequential intermediate step.
- **Goal-driven verification** can happen when the agent proposes completion and a harness-owned
  gate checks whether the current artifact satisfies the goal.

Image-to-code is the first executable task package. It is not the intended boundary of the
generic harness.

## 2. Frozen Experimental Contract

V0.1 used two independent treatment axes:

| Condition | Concrete name | Harness hard gate | Verification policy prompt |
| --- | --- | ---: | ---: |
| `H0P0` | Natural baseline | off | off |
| `H0P1` | Prompt-driven spontaneous verification | off | on |
| `H1P0` | Harness-forced verification | on | off |
| `H1P1` | Combined verification | on | on |

Important controls:

- `H` controls a harness-owned gate at the completion boundary.
- `P` controls whether the system prompt asks the agent to verify consequential claims.
- `invoke_model` remains available in P0. The P axis changes use policy, not tool availability.
- H0 has a `max_turns=500` infrastructure guard, not a normal semantic stopping rule.
- H1 has no semantic turn cap and uses `max_epochs=6` for the outer repair loop.
- The online gate returns grounded repair feedback and no agent-facing score.
- A required H1 component that is unavailable retries in place and then terminates as
  infrastructure failure. It cannot accept or incorrectly reopen the agent.

## 3. Model Invocation And Evidence Contract

V0.1 exposes one general isolated `invoke_model` interface instead of many task-specific model
tools. Its capability vocabulary includes visual locate, inspect/OCR, align, compare, image
generate/edit, and custom calls. Versioned recipes provide more detailed instruction patterns.

Confirmed evidence rules:

- The invoked model receives only explicit instruction, inputs, model profile, and output
  contract. It does not inherit the main transcript.
- A required image must be attached as image bytes through the declared input contract. Mentioning
  a path in instruction text is not image input.
- Tool transport success, valid JSON, or fluent text is not enough for evidence admission.
- Plan-only responses, irrelevant OCR, missing verdicts, guessed content, and schema-valid but
  claim-irrelevant outputs are `insufficient`.
- Every successful observation is associated privately with exact input hashes and the artifact
  version at call time. Old candidate observations do not prove a modified artifact.
- The agent may improve a call by changing the question, crop, capability, recipe, or model even
  when it reuses the same underlying model.

## 4. Harness, Gate, Trace, And Evaluation

Implemented V0.1 infrastructure includes:

- a standalone Python 3.12 Claude Agent SDK harness;
- four-condition experiment manifests and independent per-cell workspaces;
- a persistent Agent SDK session plus the H0/H1 outer controller;
- explicit `invoke_model` and `propose_finish` MCP tools;
- a copied image-to-code task package and v-harness-compatible gate backend;
- strict artifact, observation, evidence, claim, assessment, and event schemas;
- separate model transcript and structured append-only event trace;
- separate agent model-call and harness-private gate model-call accounting;
- structured online claims with stable object identity;
- immutable per-attempt artifact, render, claim-bank, feedback, and gate audits;
- exact observation provenance and freshness bookkeeping;
- a copied V3-style offline evaluator that runs after every agent has stopped.

The latest accumulated claim bank is the current H1 acceptance standard, but each historical
attempt keeps its own full bank and artifact snapshot. Later claims must not be projected back
onto earlier epochs.

The offline evaluator is not part of H1 treatment. It gives no feedback and controls no epoch.
Its four dimensions are Canvas 10%, Text Geometry 40%, Fresh Visual Bank 30%, and Structure Audit
20%, producing a weighted score from 0 to 5 with a strict pass line at 4.0.

## 5. Frozen Case1 Record

| Condition | Total | Canvas | Text Geometry | Fresh Visual | Structure |
| --- | ---: | ---: | ---: | ---: | ---: |
| Natural baseline `H0P0` | 2.44 | 5.00 | 0.00 | 4.38 | 3.12 |
| Prompt-only `H0P1` | 2.62 | 5.00 | 0.00 | 5.00 | 3.12 |
| Harness-only `H1P0` | 4.08 | 5.00 | 3.64 | 5.00 | 3.12 |
| Combined `H1P1` | 3.26 | 5.00 | 1.59 | 5.00 | 3.12 |

This is a one-case pipeline audit, not a general verification-effect result.

The defensible positive statement is narrow: harness-only produced positive evidence on the
frozen case1 artifact. V0.1 does not prove that verification generally improves final quality.

## 6. Validity Limits Found By The Audit

1. In the frozen run, the agent-facing router interpreted historical `kind:image` descriptors as
   text. P1 calls did not receive image bytes. Therefore H0P1 and H1P1 cannot measure tool-driven
   multimodal verification. The router was fixed and separately smoke-tested, but the historical
   results remain frozen and invalid for that claim.
2. The H1 private gate used a separate path-to-data-URI transport, so it did receive images. Its
   judgments can still contain semantic inconsistency, ambiguous claims, or verifier-specific
   bias.
3. Text Geometry uses greedy one-to-one text binding. Punctuation, whitespace, line breaks,
   repeated strings, and Unicode differences can be reported as missing text even when visible
   content exists.
4. The visual claim bank expanded across epochs. This is auditable, but it can move the acceptance
   contract and push artifacts toward verifier-specific optimization.
5. There is only one evaluated case. No cross-case stability, task generality, or causal claim is
   established.

## 7. Verification Episode And RL Direction

V0.1 accepted a `verification_episode_v1` analysis unit that separates:

- observation quality;
- the agent's response to the observation;
- the resulting artifact effect.

Transcript and event trace stay separate so episodes can be reconstructed offline without asking
the agent to self-report bookkeeping. The intended long-term path is to use strong harness-forced
verification-and-repair trajectories for cold start, then test whether a prompted or trained model
spontaneously chooses useful verification. No RL training or capability internalization was
completed in V0.1.

## 8. Research Interface Delivered

V0.1 is registered on the main board with:

- a dedicated Chinese summary page;
- complete collapsible English system-prompt modules;
- reader-only Chinese translations with an English/Chinese switch;
- the four conditions and four evaluator dimensions kept visibly separate;
- case1 scores displayed with evidence-validity caveats;
- the case1 four-condition trajectory package available in the shared viewer.

The English prompt files remain the runtime authority. Chinese prompt files are reading
translations only.

## 9. Unfinished V0.1 Work

The following items were known but are not automatically V0.2 requirements:

- rerun the P1 cells after the image-attachment repair;
- calibrate the online verifier against human-labeled disputed claims;
- replace or audit greedy Text Geometry matching;
- add infrastructure wall-time and model-call budgets;
- add structured context-threshold reset support;
- add a dedicated provider adapter for image generation and editing;
- complete stable 3-case smoke before any large comparison run.

V0.2 must decide which of these still belong after its architecture changes.

## 10. Freeze Rule

V0.1 scores, prompts, run interpretations, and known defects remain historical records. New runs
must use new run IDs and must not overwrite the frozen case1 result. Architectural changes are
made in V0.2 rather than mutating this version until the original experiment can no longer be
reconstructed.