Skip to content

Running real extractors (GPU runbook)

The same harness, with a real extractor instead of the simulated one (issue #3). Reproduces the local results directory.

Hard-won environment notes (read first)

These were discovered running on RunPod; they save hours:

  • PaddleOCR needs a GPU architecture paddle supports. PaddlePaddle's precompiled wheels have no kernels for Blackwell (sm_120, e.g. RTX PRO 6000 / 5090) — you get RuntimeError: Unsupported GPU architecture. Use Ampere (A100 sm_80, A5000/A6000 sm_86) or Ada (RTX 4090 / L4 / L40 sm_89).
  • Pin paddlex==3.1.0 to match paddleocr==3.1.0, or the constructor dies with PaddlePredictorOption.__init__() takes 1 positional argument.
  • paddlex==3.1.0 eagerly imports its RAG retriever, which uses the removed langchain.docstore / langchain.text_splitter (gone in langchain ≥0.2). Pinning langchain<0.2 no longer resolves (it conflicts with paddlex's own deps → ResolutionImpossible). Since OCR never uses that retriever, shim the two moved modules to their new homes instead (see the LANGCHAIN SHIM block in Setup) — a one-liner that makes from paddleocr import PaddleOCR succeed.
  • RapidOCR is architecture-independent (ONNX Runtime) — it runs anywhere, CPU or GPU, and is the most robust real-OCR option. Prefer it when the GPU is exotic.
  • dots.ocr loads with transformers==4.51.3 + qwen_vl_utils, but raw transformers.generate fights its custom model; its intended runtime is vllm serving — wire the adapter to a vllm endpoint (issue #3).

Setup

git clone https://github.com/bhaskargurram-ai/verifydoc && cd verifydoc
python3.11 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev,data]"

# Option A — RapidOCR (recommended, architecture-independent)
pip install rapidocr onnxruntime          # or onnxruntime-gpu for speed

# Option B — PaddleOCR (needs a paddle-supported GPU)
pip install "paddlepaddle-gpu==3.1.0" -i https://www.paddlepaddle.org.cn/packages/stable/cu126/
pip install "paddleocr==3.1.0" "paddlex==3.1.0"   # NO langchain pin — it conflicts

# LANGCHAIN SHIM — paddlex 3.1.0's retriever imports two removed langchain modules;
# alias them to their new homes so `from paddleocr import PaddleOCR` works (OCR
# never uses the retriever). langchain-community + langchain-text-splitters come in
# transitively with paddlex.
SP=$(python -c 'import langchain,os;print(os.path.dirname(langchain.__file__))')
mkdir -p "$SP/docstore"
echo 'from langchain_community.docstore.document import Document' | tee "$SP/docstore/__init__.py" > "$SP/docstore/document.py"
echo 'from langchain_text_splitters import *' > "$SP/text_splitter.py"

Verified on an RTX 4090 (Ada, sm_89), paddle 3.1.0, torch 2.4.1+cu124.

Smoke test one receipt

python - <<'EOF'
from benchmark.datasets import cord
from verifydoc.adapters import get_adapter

item = cord.load(split="validation", limit=1, with_images=True)[0]
adapter = get_adapter("rapidocr")          # or "paddleocr-vl"
for p in adapter.extract(item.doc, item.schema):
    print(p.path, p.value, p.meta.get("token_logprobs"))
EOF

Full run

python scripts/run_benchmark.py --config configs/cord-rapidocr.yaml   --out paper/generated/cord-rapidocr
python scripts/run_benchmark.py --config configs/funsd-rapidocr.yaml  --out paper/generated/funsd-rapidocr
python scripts/run_benchmark.py --config configs/cord-paddleocr.yaml  --out paper/generated/cord-paddleocr   # paddle GPU
python scripts/run_benchmark.py --config configs/funsd-paddleocr.yaml --out paper/generated/funsd-paddleocr

# Multi-extractor ensemble: run several extractors on the same slice and
# adjudicate per field (agreement + best grounding) — reports each single
# extractor vs the ensemble. Add api-vlm (needs ANTHROPIC_API_KEY) for diversity.
python scripts/ensemble_experiment.py --dataset cord --limit 60 \
    --extractors rapidocr,paddleocr-vl --out paper/generated/ensemble-cord.json

The harness dispatches extractor: through the adapter registry (mock | text-search | rapidocr | paddleocr-vl | dots-ocr | docling | api-vlm) and dataset: (synthetic | cord | funsd). CORD/FUNSD images are exported automatically (with_images).

What to report

Pin model versions in the table header (rapidocr==x, paddleocr==3.1.0, GPU + driver). Results feed the paper via make results; see the local make results output for the current numbers and reading.

Cloud

Any Ampere/Ada single-GPU pod (RTX 4090 ≈ $0.4–0.7/hr) runs everything except Blackwell-only cards for paddle. The full sweep is an hours-not-days job. ```