# VDiff PDF-to-HTML Data Generation Loop

Source snapshot: `ChengJade/VDiff`, imported from the uploaded `VDiff-main.zip`.

This project preserves a mentor implementation of a PDF-to-HTML data generation and verification flow. The main pipeline is documented in `data_generation_loop/README.md`.

## What This Contains

- `data_generation_loop/`: the primary 5-step PDF-to-HTML pipeline.
- `data_generation_loop/testcase/`: pipeline input cases, each with PDF/page snapshot assets.
- `data_generation_loop/output-test/` and `data_generation_loop/output-demo/`: existing generated outputs.
- `rubrics-diff/`: rubric comparison/evaluation utilities.
- `viewer/`: small HTML viewers for bbox/layout inspection.
- `testcase/`, `output/`, `run_vlm_diff.py`, `run_all_cases.sh`: top-level VDiff comparison harness and sample cases.

## Pipeline Summary

The main flow converts PDF pages into visually matched HTML and evaluates the result:

1. Transcribe: generate an initial HTML reproduction from a PDF screenshot.
2. Bind: map extracted PDF text/images onto generated HTML elements.
3. Calibrate: tune font size and line height to match the source PDF.
4. Validate: compare rendered HTML bbox/layout measurements against PDF-derived ground truth.
5. Fix and polish: use Claude CLI plus visual rubrics to iteratively improve CSS/layout.

## Entry Points

- Full instructions: `data_generation_loop/README.md`
- Detailed Chinese pipeline/deployment guide: `data_generation_loop/PIPELINE_AND_DEPLOYMENT_GUIDE.zh.md`
- Web UI server: `data_generation_loop/demo/server.py`
- Local service setup: `data_generation_loop/scripts/setup_local_service.sh`
- Local service start: `data_generation_loop/scripts/start_local_service.sh`
- CLI scripts: `data_generation_loop/scripts/run_pipeline.sh` and `data_generation_loop/scripts/run_full_pipeline.sh`
- Core spatial binding code: `data_generation_loop/spatial_binding/`
- Evaluation code: `data_generation_loop/eval/`

## Existing Results

The reproduction package already contains historical outputs, but they live in different directories with different meanings:

- `data_generation_loop/output-test/{case}`: mentor historical outputs. These are the best place to study previous completed examples because most cases include `transcribed.html`, `bound.html`, `1_font_fixed.html`, `dashboard-verify.html`, `1_css_fixed.html`, and rubric results.
- `data_generation_loop/output-demo/{job_id}`: outputs from PDFs uploaded through the demo service. For example, `cbe23568` is an uploaded run with five CSS fixing iterations.
- `data_generation_loop/output-gallery/{case}`: current gallery run workspace. It can be cleared and regenerated by `Run` / `Run All`, so a gallery card marked `partial` only means the gallery workspace does not currently have a final `1_css_fixed.html`; it does not mean the mentor historical result is missing.

The live demo now exposes a `Past mentor results` section on the home screen. Use `Open timeline` there to replay an old case through the pipeline viewer without rerunning it, or expand `Artifacts` to open the target screenshot, final HTML, verify dashboard, and rubric JSON directly.

## Notes

- The pipeline expects an OpenAI-compatible API proxy by default at `http://localhost:4151`.
- Step 5 depends on an authenticated Claude CLI.
- Rendering and validation depend on Playwright/Chromium.
- Font calibration depends on Node.js and the `canvas` package under `data_generation_loop/calibration/`.
- This snapshot was imported as-is, including existing outputs, cache-like bytecode, and bundled `node_modules`, so it remains faithful to the uploaded GitHub archive.