Paperdelta

An accuracy result changes from 84.1% to 80.9%. The abstract, table and appendix still repeat the old number; a 3.1 percentage point improvement and an outperforms baseline claim no longer hold. PaperDelta traces those statements to declared experiment evidence and brings the affected locations together for review, even when no manuscript file changed.

Local Python CLI · LaTeX / Markdown / Quarto + optional Word/PDF · exact decimal arithmetic · offline HTML · optional MCP. Checking needs no model key, GPU or TeX installation.

A 27-second walkthrough of real reports: change experiment evidence, inspect the abstract, table and claim, then open a separate source/PDF example with a stale export. Open the full-size animation · View the static screenshot.

2.0.0 brings the declared experiment identity, exact source hashes, original context, units, records and conflicts into one Agent proposal review. Studio and the CLI share that evidence; staged agents can retract a mistaken declaration within a fixed budget. Mappings still require explicit review and confirmation. Agent evidence review · Incremental checks · Annotated-copy guide · Choose your task · Revision workbench · Notebook/Quarto workflows.

Use static LaTeX, Markdown/Quarto, Word or PDF with the same declared experiment evidence and statistical contracts. Run paperdelta studio for local binding review, snapshots, ongoing checks and draft recovery. The interface supports English/Chinese throughout; see the Studio guide and delivery roadmap. The project has received positive informal feedback from several users; it is not presented as a measured usability study. Releases use normal version numbers.

Install and see the first report

For static source examples, use paperdelta demo --document markdown --out md-demo --open or paperdelta demo --document quarto --out qmd-demo --open. No extra or renderer is needed.

For the source/PDF example, install paperdelta[pdf] and run paperdelta demo --document pdf --out pdf-demo --open. The report highlights original pages; PDF review is read-only and does not perform OCR.

For a Word example, install paperdelta[docx] and run paperdelta demo --document docx --out word-demo --open. Word review is read-only; supported structures and positions are explicit.

Use Python 3.11 or newer. In a virtual environment:

python -m pip install paperdelta
paperdelta demo --out paperdelta-demo --open

With uv installed, try uvx --python 3.12 paperdelta@2.0.0 demo --out paperdelta-demo --open in an isolated tool environment. Use a new output directory for each run. See optional native readers and startup diagnostics.

If needed, create the environment first with python -m venv .venv. Activate it with .venv\Scripts\Activate.ps1 on Windows PowerShell or source .venv/bin/activate on macOS/Linux (use python3 when creating it there).

The installed package includes the complete offline example. Open paperdelta-demo/review/report.html if it does not open automatically. Choose English or Chinese in the report. The default scenario intentionally finds stale numbers, a false comparison and changed figure dependencies. Related numeric edits are withheld until the conclusion is addressed. No paper changes are applied.

--scenario baseline shows unchanged evidence; --scenario safe-update prepares four numeric edits and a readable diff. Each run needs a new --out directory. The demo command returns 0 on successful creation; its report retains the actual check result, normally 1 for the changed scenario. See demo details.

A wheel, source archive and separately licensed evaluation bundle are also available from GitHub Releases. Verify the wheel with the release's SHA256SUMS, then run python -m pip install ./paperdelta-2.0.0-py3-none-any.whl.

Connect an existing paper

Run paperdelta studio in the paper folder to create a configuration, declare evidence, select locations and explicitly confirm bindings in a local browser. No hand-written YAML is needed for this workflow. Start with the workbench guide. The CLI workflow remains available:

paperdelta init --paper paper/main.tex --data results/metrics.csv
paperdelta batch guide
paperdelta check --report build/paperdelta

init discovers inputs without accepting bindings. The batch guide reuses explicit source, record identity, units, aggregation and seed choices across a declared results table. It suggests locations from context, lets you choose repeated occurrences, and shows the final selection before accept. Equal numbers alone are not evidence of identity. JSON sources and individual/derived metrics use paperdelta guide.

Start with a selected file or region through scope set; record a justified non-result number through scope exclude. Both preview before --accept. Changes to excluded text require review, and accepted failing bindings cannot be hidden by changing scope. See batch binding and coverage, guided binding and repairs, and the complete JSON tutorial.

Use --lang zh-CN before a subcommand or save settings --language zh-CN. Language preferences do not change scientific results or evidence identity.

Review the next experiment

paperdelta snapshot create submitted-v1
# Run experiments in your existing workflow.
paperdelta watch --baseline submitted-v1 --report build/paperdelta --open

Saving data, paper or configuration triggers a complete check after a short quiet period. The report marks observed changes as pending until rechecked and refreshes in the browser. Its review queue identifies evidence issues, broken bindings, conclusions, numeric updates and figures. Stop with Ctrl+C. A one-off check remains available with check --baseline submitted-v1.

paperdelta fix --report build/paperdelta/report.json --out changes.pdpatch.json
paperdelta apply changes.pdpatch.json --dry-run
paperdelta apply changes.pdpatch.json --write

Only verified numeric locations enter a patch. Changed inputs, ambiguous or overlapping edits are refused; invalid related claims block numeric changes. A transaction preserves original bytes. paperdelta recover TRANSACTION_ID previews recovery; add --write to restore when no later author edits intervene. The checker never runs experiment or plotting scripts.

The CI adapter compares the target commit and reports changes in bindings, rules, scope and review declarations alongside consistency results.

Work with an Agent

python -m pip install 'paperdelta[mcp]'
paperdelta -C /path/to/paper-project mcp

Twenty-two optional MCP tools provide checks, evidence, proposals and repairs. Four batch-session tools let an Agent select candidate IDs while the program assembles the proposal. Sessions are in memory, bounded and invalidated by changed inputs. The tools do not accept mappings or write paper files. The Agent guide includes the workflow and host configuration. CLI JSON and schemas remain available without MCP.

The new bounded mapping session returns typed errors and correction hints, keeps the last valid draft and allows explicit abstention. Deterministic protocol tests verify tool behavior. In the v1.4 real-model study, both the first run and its separately frozen correction achieved 0/9 complete mappings and 0/3 required abstentions. Automatic mapping remains experimental.

Supported scope

• Exit 0: required accepted checks agree with the supplied evidence.
• Exit 1: an accepted check fails.
• Exit 2: evidence, configuration or parsing is incomplete, no bindings exist, or a live check is pending.

Reports keep unbound, excluded, outside-scope and unsupported content visible. require_complete_coverage can require numeric coverage in the declared scope; existing bindings always run. Agreement is not certification of scientific truth.

Supported inputs include literal LaTeX input/include, declared literal macro arguments, bounded literal table cells, CSV/TSV/JSON, static Excel and experiment exports, explicit aggregation, derived values with units, limited comparisons and figure provenance. Dynamic TeX, arbitrary macro expansion, automatic significance claims and global SOTA verification remain outside the supported contract. Word paragraphs and ordinary tables are supported with paperdelta[docx]; revisions, fields and complex layouts remain unverified. See rules and limits and the Word boundary.

Text PDFs use optional paperdelta[pdf], native page/box coordinates and explicit extraction identities. Scanned pages and unreliable layouts remain unverified. Multiple manuscripts share explicitly declared metrics, with optional source/export relations. See the PDF boundary and workflow.

Cross-platform CI covers Python 3.11–3.14 on Windows, Linux, Apple Silicon and Intel macOS. Historical validation, native Mac evidence and current release checks distinguish what each run established.

Development

git clone https://github.com/amos689/paperdelta.git
cd paperdelta
python -m pip install -e '.[dev,mcp]'
python tools/run_tests.py -q
python -m ruff check src tests tools
python -m ruff format --check src tests tools
python tools/check_docs.py

Contributing · Changelog · MIT license · Third-party notices · Security · Mac installation