Skip to content

README.md

Latest commit

 

History

History
274 lines (211 loc) · 6.66 KB

README.md

File metadata and controls

274 lines (211 loc) · 6.66 KB
Raw

OCR Model Support

Table of Contents

  1. dots.ocr Support
  2. MinerU Support
  3. Paddler-OCR Support
  4. DeepSeek-OCR Support

1. dots.ocr Support

To launch dots.ocr, follow the instructions in Launching the Serving Service, specifying the dots.ocr model, setting the model path to /llm/models/dots.ocr, the served-model-name to model, and the port to 8000.

Once the service is running, you can use the method provided in the dots.ocr repository to launch Gradio for testing.

Clone the repository

git clone https://github.com/rednote-hilab/dots.ocr.git
cd dots.ocr

Install dependencies

pip install -e . --no-deps
pip install gradio gradio_image_annotation PyMuPDF qwen_vl_utils

Launch Gradio for testing

python demo/demo_gradio.py 9000

You can refer to the dots.ocr guide for more details.

2. MinerU 2.6 Support

This guide shows how to launch the MinerU 2.6 model using the vLLM inference backend.

Start the MinerU Service

Set up the environment variables and launch the vLLM API server:

export MODEL_NAME="/llm/models/MinerU2.5-2509-1.2B/"
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export VLLM_OFFLOAD_WEIGHTS_BEFORE_QUANT=1

python3 -m vllm.entrypoints.openai.api_server \
  --model $MODEL_NAME \
  --dtype float16 \
  --enforce-eager \
  --port 8000 \
  --host 0.0.0.0 \
  --trust-remote-code \
  --gpu-memory-util 0.85 \
  --no-enable-prefix-caching \
  --max-num-batched-tokens=32768 \
  --max-model-len=32768 \
  --block-size 64 \
  --max-num-seqs 256 \
  --served-model-name MinerU \
  --tensor-parallel-size 1 \
  --pipeline-parallel-size 1 \
  --logits-processors mineru_vl_utils:MinerULogitsProcessor

💡 Notes

  • --logits-processors mineru_vl_utils:MinerULogitsProcessor enables MinerU’s custom post-processing logic.

Run the demo

  1. To verify mineru
#mineru -p <input_path> -o <output_path> -b vlm-http-client -u http://127.0.0.1:8000
mineru -p /llm/MinerU/demo/pdfs/small_ocr.pdf -o ./ -b vlm-http-client -u http://127.0.0.1:8000
  1. Using by gradio
# refer to http://your_ip:8002/?view=api for gradio's api guide
mineru-gradio --server-name 0.0.0.0 --server-port 8002
from gradio_client import Client, handle_file

client = Client("http://localhost:8002/")
result = client.predict(
    file_path=handle_file('/llm/MinerU/demo/pdfs/small_ocr.pdf'),
    end_pages=500,
    is_ocr=False,
    formula_enable=True,
    table_enable=True,
    language="ch",
    backend="vlm-http-client",
    url="http://localhost:8000",
    api_name="/to_markdown"
)
print(result)

You can refer to the MinerU usage guide for more details.

3. Paddler-OCR Support

Start vLLM Service

ZE_AFFINITY_MASK=6 \
vllm serve --model /llm/models/LLM2/PaddleOCR-VL \
    --served-model-name PaddleOCR-VL-0.9B \
    --trust-remote-code \
    --max-num-batched-tokens 16384 \
    --no-enable-prefix-caching \
    --mm-processor-cache-gb 0 \
    --enforce-eager

Install Paddle Dependencies

pip install "paddleocr[doc-parser]" paddlepaddle
paddlex --install serving

Deploy Paddle Service

  • generate configuration
# include PP-DocLayoutV2 and PaddleOCR-VL-0.9B by default
paddlex --get_pipeline_config PaddleOCR-VL
  • replace native backend to vllm server
SubModules:
    LayoutDetection:
        module_name: layout_detection
        model_name: PP-DocLayoutV2
        ...
    VLRecognition:
        ...
        # replaced part
        genai_config:
            backend: vllm-server
            server_url: http://127.0.0.1:8000/v1
  • start paddlex service
paddlex --serve --pipeline ./PaddleOCR-VL.yaml

Run the demo

1.Call vLLM Service Need to use the specified format to use paddleocr. 
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url="http://localhost:8000/v1",
    timeout=3600
)

# Task-specific base prompts
TASKS = {
    "ocr": "OCR:",
    "table": "Table Recognition:",
    "formula": "Formula Recognition:",
    "chart": "Chart Recognition:",
}

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image_url",
                "image_url": {
                    "url": "https://ofasys-multimodal-wlcb-3-toshanghai.oss-accelerate.aliyuncs.com/wpf272043/keepme/image/receipt.png"
                }
            },
            {
                "type": "text",
                "text": TASKS["ocr"]
            }
        ]
    }
]

response = client.chat.completions.create(
    model="PaddleOCR-VL-0.9B",
    messages=messages,
    temperature=0.0,
    max_tokens=128,
)
print(f"Generated text: {response.choices[0].message.content}")
2.Call Paddle Service 
import requests
request_data = {
    "file": "https://ofasys-multimodal-wlcb-3-toshanghai.oss-accelerate.aliyuncs.com/wpf272043/keepme/image/receipt.png",
    "fileType": 1,
    "useLayoutDetection": True, # default value is True, used for layout_det_res
}

response = requests.post(
    url="http://localhost:8080/layout-parsing",
    json=request_data,
    timeout=3600
)

# print result from PaddleOCR-VL-0.9B
print(response.json()['result']['layoutParsingResults'][0]['markdown']['text'])

# print result from PP-DocLayoutV2
print(response.json()['result']['layoutParsingResults'][0]['prunedResult']['layout_det_res'])
3.Offline Paddle Service

Use PP-DocLayoutV2 model offline.

from paddleocr import PaddleOCRVL

doclayout_model_path = "/path/to/your/PP-DocLayoutV2/"

pipeline = PaddleOCRVL(vl_rec_backend="vllm-server", 
                       vl_rec_server_url="http://localhost:8000/v1", 
                       layout_detection_model_name="PP-DocLayoutV2",  
                       layout_detection_model_dir=doclayout_model_path)

output = pipeline.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/paddleocr_vl_demo.png")

for i, res in enumerate(output):
    res.save_to_json(save_path=f"output_{i}.json")
    res.save_to_markdown(save_path=f"output_{i}.md")

You can refer to the vLLM Paddler-OCR guide for more details.

4. DeepSeek-OCR Support

You can refer to here to know how to use.