baidu/Unlimited-OCR-2
类型 image-text-to-text · 框架 transformers · 下载 147,588 · 收藏 562
类型 image-text-to-text · 框架 transformers · 下载 147,588 · 收藏 562
定位:百度发布的 Unlimited OCR Works 2 是面向超长文档解析的视觉语言模型,在有限上下文窗口内完成多页与上百页文档的连续识别。 核心能力
infer_multi 一次请求解析整份文档的所有页面,按页序输出对应 <PAGE>,可利用跨页上下文。infer 的动态切图(gundam 模式),多页每页编码为一个全局视图(base 模式)。适用人群:适合需要批量处理合同、档案、书籍等长文档 OCR 的开发者,需 Python 3.10、torch 2.13、transformers 4.52.1 及 NVIDIA GPU 环境。 注意:官方在 A800-80GB 上测试,其他显存配置未说明。
位置づけ:Baidu が公開した Unlimited OCR Works 2 は、超長文書の解析を想定した視覚言語モデルで、限られたコンテキスト長の中で数十ページから百ページ規模の文書を連続して読み取ります。 主な機能
infer_multi は文書全体のページを 1 リクエストで解析し、ページ順に <PAGE> を出力するため、ページ間の文脈を利用できます。infer の動的タイル(gundam モード)、複数ページは各ページを 1 つの全体ビュー(base モード)として符号化します。対象となる利用者:契約書や档案、書籍など長い文書の OCR をまとめて処理したい開発者向け。Python 3.10、torch 2.13、transformers 4.52.1、NVIDIA GPU 環境が必要です。 注意:動作確認は A800-80GB のみで、他の VRAM 構成は明記されていません。
정체:바이두가 공개한 Unlimited OCR Works 2는 초장문 문서 파싱을 위한 비전-언어 모델로, 제한된 컨텍스트 창 안에서 수십 쪽에서 수백 쪽 문서를 연속으로 인식합니다. 핵심 기능
infer_multi는 문서 전체 페이지를 한 번의 요청으로 파싱하고 페이지 순서대로 <PAGE>를 출력해 페이지 간 문맥을 활용합니다.infer의 동적 타일링(gundam 모드), 다중 페이지는 각 페이지를 하나의 전역 뷰(base 모드)로 인코딩합니다.대상 사용자:계약서, 기록물, 서적처럼 긴 문서의 OCR을 일괄 처리하려는 개발자에게 적합합니다. Python 3.10, torch 2.13, transformers 4.52.1, NVIDIA GPU 환경이 필요합니다. 주의:A800-80GB에서만 검증되었고 다른 VRAM 구성은 명시되지 않았습니다.
What it is: Unlimited OCR Works 2 is Baidu's vision-language model for parsing very long documents, reading dozens to hundreds of pages in one pass within a fixed context window. Key capabilities
infer_multi parses every page of a document in one request and returns one <PAGE> per page in input order, so cross-page context is available.infer with dynamic tiling (gundam mode); multi-page runs encode each page as one global view (base mode).Who it is for: Developers batch-processing long documents such as contracts, archives, and books. Requires Python 3.10, torch 2.13, transformers 4.52.1, and an NVIDIA GPU. Caveats: Tested on an A800-80GB; other GPU memory configurations are not documented.
来源 · 查看原文
pipeline_tag: image-text-to-text language:
tags:
license: mit library_name: transformers ---



Unlimited OCR 2 supports Transformers (native, for quick tests) and SGLang (recommended for serving).
Tested with Python 3.10, torch==2.13.0, transformers==4.52.1, huggingface-hub==0.36.0, tokenizers==0.21.4, on an NVIDIA A800-80GB:
pip install torch==2.13.0 torchvision pillow transformers==4.52.1 huggingface-hub==0.36.0 \
tokenizers==0.21.4 addict==2.4.0 einops==0.8.2 easydict==1.13
Load the model once:
import torch
from transformers import AutoModel, AutoTokenizer
model_name = "baidu/Unlimited-OCR-2"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True, use_safetensors=True,
torch_dtype=torch.bfloat16).eval().cuda()
Multi-page inference (infer_multi). All pages of a document are parsed in one request, so the model can use cross-page context. Pass the page images in page order:
pages = [f"your_doc/page_{i}.jpg" for i in range(1, 8)]
text, num_tokens = model.infer_multi(tokenizer, image_files=pages,
output_path="output/multi", save_results=True)
The prompt is built automatically as "<PAGE><image>" * num_pages + "Multi page parsing.", and each page is encoded as one global view (base mode). The output contains one <PAGE> per input page, in the same order as image_files.
Single-page inference (infer). A single page is encoded with dynamic tiling (gundam mode) by default:
text = model.infer(tokenizer, image_file="your_doc/page_1.jpg",
output_path="output/single", save_results=True)
Set up the environment (uv-managed virtualenv, Python 3.10) and install the customized SGLang wheel sglang-0.5.20.dev13+g75b9b06b02-py3-none-any.whl, plus PyMuPDF for PDF-to-image conversion:
uv venv --python 3.10
source .venv/bin/activate
uv pip install wheel/sglang-0.5.20.dev13+g75b9b06b02-py3-none-any.whl
uv pip install pymupdf==1.27.2.2
uv pip install fastokens==0.3.2
Start the SGLang server:
# Page-window scheduling for very long documents (the KV tail of the previous window is reused).
export SGLANG_UOCR_EANABLE_DOC_LIMIT=1 # 1 = enable, 0 = whole document in one window
export SGLANG_UOCR_DOC_GUNDAM_PAGE_WINDOW=10 # pages per window (gundam mode)
export SGLANG_UOCR_DOC_BASE_PAGE_WINDOW=10 # pages per window (base mode)
export SGLANG_UOCR_MAX_PAGE_TOKENS=20000 # per-page output cap; must be > 0
# Chunk cap for the SAM attention bias. The built-in default (512) allows a single
# ~462 MiB allocation and can OOM; 64 is equivalent with a lower peak.
export SGLANG_OPT_UOCR_SAM_REL_POS_BIAS_MB=64
# Pass large crop tensors through /dev/shm instead of pickling them into the
# tokenizer -> scheduler frame.
export SGLANG_UOCR_SHM_WRAP_MODEL_DATA=1
export SGLANG_UOCR_SHM_WRAP_MIN_MB=1
export SGLANG_VLM_CACHE_SIZE_MB=0
# Reduces fragmentation failures on the large contiguous allocations.
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
python -m sglang.launch_server \
--model-path baidu/Unlimited-OCR-2 \
--served-model-name Unlimited-OCR-2 \
--chat-template unlimited-ocr \
--tokenizer-backend fastokens \
--tokenizer-worker-num 8 \
--mem-fraction-static 0.7 \
--max-running-requests 64 \
--max-prefill-tokens 2048 \
--enable-custom-logit-processor \
--enable-streaming-session \
--allow-auto-truncate \
--disable-radix-cache \
--mm-preprocess-cache-size-mb 0 \
--cuda-graph-backend-decode full --cuda-graph-max-bs-decode 64 \
--cuda-graph-backend-prefill full \
--cuda-graph-max-bs-prefill 16384 \
--cuda-graph-bs-prefill 1024 2048 \
--cuda-graph-config '{"prefill": {"backend": "full", "full_prefill_max_req": 64}}' \
--host 0.0.0.0 \
--port 10000
Notes on the knobs that matter for memory:
--mem-fraction-static is the share reserved for weights + KV. 0.75 leaves too little headroom at 32+ concurrent requests; 0.7 is stable.--max-prefill-tokens drives a temporary tensor of roughly [tokens, context/32], so memory grows linearly with it: 4096 -> 2048 halves that peak.--cuda-graph-bs-prefill bucket must equal --max-prefill-tokens. A smaller one makes oversized batches fall back to eager; a larger one is never used and only wastes capture memory.--cuda-graph-config's full_prefill_max_req is the request-slot count for the full-prefill CUDA graph. The default (8) makes 16/32 concurrency fall back to eager.To parse a whole document in one window instead (SGLANG_UOCR_EANABLE_DOC_LIMIT=0).
Send requests to the OpenAI-compatible API:
import base64
import json
import os
import tempfile
import fitz
import requests
from sglang.srt.sampling.custom_logit_processor import DeepseekOCRNoRepeatNGramLogitProcessor
server_url = "http://127.0.0.1:10000"
NGRAM_PROCESSOR = DeepseekOCRNoRepeatNGramLogitProcessor.to_str()
session = requests.Session()
session.trust_env = False
def pdf_to_images(pdf_path, dpi=300):
doc = fitz.open(pdf_path)
tmp_dir = tempfile.mkdtemp(prefix="pdf_ocr_")
mat = fitz.Matrix(dpi / 72, dpi / 72)
image_paths = []
for i, page in enumerate(doc):
image_path = os.path.join(tmp_dir, f"page_{i + 1:04d}.png")
page.get_pixmap(matrix=mat).save(image_path)
image_paths.append(image_path)
doc.close()
return image_paths
def encode_image(image_path):
ext = os.path.splitext(image_path)[1].lower()
mime = "image/jpeg" if ext in (".jpg", ".jpeg") else f"image/{ext.lstrip('.')}"
with open(image_path, "rb") as f:
data = base64.b64encode(f.read()).decode("utf-8")
return {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{data}"}}
def build_content(image_paths):
prompt = "<PAGE><image>" * len(image_paths) + "Multi page parsing."
return [encode_image(path) for path in image_paths] + [{"type": "text", "text": prompt}]
def generate(image_paths, crop_mode=True, tokens_per_page=8000):
# A wider n-gram window for multi-page documents, as in our evaluation setup.
window_size = 1024 if len(image_paths) > 1 else 128
payload = {
"model": "Unlimited-OCR-2",
"messages": [{"role": "user", "content": build_content(image_paths)}],
# Must be positive: it is the global output budget for the whole document.
"max_tokens": tokens_per_page * len(image_paths),
"temperature": 0,
"skip_special_tokens": False,
"images_config": {"crop_mode": crop_mode}, # True = gundam (tiles), False = base
"custom_logit_processor": NGRAM_PROCESSOR,
"custom_params": {
"ngram_size": 35,
"window_size": window_size,
"page_token_id": 128827, # <PAGE> marker id (= sparse_attention_config.page_token_id)
"eos_token_id": 1, # = eos_token_id in config.json
"max_page_tokens": 20000, # per-page output cap, <= 0 disables the page guard
},
"stream": True,
}
response = session.post(
f"{server_url}/v1/chat/completions",
headers={"Content-Type": "application/json"},
data=json.dumps(payload),
timeout=6000,
stream=True,
)
response.raise_for_status()
chunks = []
for line in response.iter_lines(decode_unicode=True):
if not line or not line.startswith("data:"):
continue
data = line[len("data:"):].strip()
if data == "[DONE]":
break
event = json.loads(data)
choices = event.get("choices") or [] # the final chunk carries usage only
if not choices:
continue
delta = (choices[0].get("delta") or {}).get("content") or ""
if delta:
print(delta, end="", flush=True)
chunks.append(delta)
return "".join(chunks)
# Multi page: all pages of a document, parsed in one request
pages = [f"your_doc/page_{i}.jpg" for i in range(1, 8)]
generate(pages)
# Single page
generate(["your_doc/page_1.jpg"])
# PDF
generate(pdf_to_images("your_doc.pdf", dpi=300))
Unlimited-horizon parsing in action: a full-length book parsed in a single request (replay).
Harry Potter replay
We would like to thank Deepseek-OCR, Deepseek-OCR-2, PaddleOCR for their valuable models and ideas, and SGLang for the serving framework.
@misc{yin2026unlimitedocrworks,
title={Unlimited OCR Works},
author={Youyang Yin and Huanhuan Liu and YY and Qunyi Xie and Chaorun Liu and Shiqi Yang and Shaohua Wang and Zhanlong Liu and Hao Zou and Jinyue Chen and Shu Wei and Jingjing Wu and Mingxin Huang and Zhen Wu and Guibin Wang and Tengyu Du and Lei Jia},
year={2026},
eprint={2606.23050},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2606.23050},
}