MiniCPMOCRDocumentConverter
Extract text from PDF files and images using a MiniCPM-OCR inference server.
MiniCPMOCRDocumentConverter detects the layout of each page in your pipeline with a PP-DocLayout model, sends the cropped blocks to a MiniCPM-OCR inference server that you host, and returns the recognized text as Markdown in Haystack Document objects. It returns one Document per source, with pages separated by form feed characters (\f). Use it in indexing pipelines to make scanned documents and images searchable.
Key Features
- Extracts text from PDF, PNG, JPEG, BMP, TIFF, and WebP files.
- Attaches optional metadata to the resulting documents. You can pass one metadata dictionary for all documents or a list with one dictionary per source.
Configuration
- Make sure you have a running MiniCPM-OCR inference server that your pipeline can reach, and note its base URL.
- Configure
MiniCPMOCRDocumentConverterin your pipeline YAML. Setserver_urlto the base URL of your server andmodelto the model name your server serves. If your server isn't vLLM, setbackendtollamacpp,ollama, oropenai. - Run the indexing pipeline on a test file and check that the resulting documents contain the recognized text. If the run fails, check that the server is running and that the pipeline can reach
server_url.
The integration is published as the minicpm-ocr-haystack package. It's already available in the runtime, so you don't need to add it to the dependencies block of your pipeline YAML.
Connections
MiniCPMOCRDocumentConverter accepts file paths or ByteStream objects as sources, and optional metadata as meta. It outputs a list of Document objects (documents).
Connect the pipeline's file input to the sources input. Connect the documents output to DocumentSplitter or DocumentWriter for further processing.
Source Code
To check this component's source code, browse the minicpm_ocr integration in the Haystack Core Integrations repository.
Usage Examples
Basic Configuration
minicpm_ocr_converter:
type: haystack_integrations.components.converters.minicpm_ocr.converter.MiniCPMOCRDocumentConverter
init_parameters:
server_url: http://your-minicpm-ocr-server:8000
model: minicpm-ocr
backend: vllm
Using the Component in a Pipeline
This indexing pipeline converts uploaded files with OCR, splits the text, and writes the chunks to a document store.
# haystack-pipeline
components:
minicpm_ocr_converter:
type: haystack_integrations.components.converters.minicpm_ocr.converter.MiniCPMOCRDocumentConverter
init_parameters:
server_url: http://your-minicpm-ocr-server:8000
model: minicpm-ocr
backend: vllm
splitter:
type: haystack.components.preprocessors.document_splitter.DocumentSplitter
init_parameters:
split_by: word
split_length: 250
split_overlap: 30
writer:
type: haystack.components.writers.document_writer.DocumentWriter
init_parameters:
document_store:
type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
init_parameters:
hosts:
- ${OPENSEARCH_HOST}
index: minicpm-ocr-demo
embedding_dim: 768
create_index: true
connections:
- sender: minicpm_ocr_converter.documents
receiver: splitter.documents
- sender: splitter.documents
receiver: writer.documents
inputs:
files:
- minicpm_ocr_converter.sources
max_runs_per_component: 100
metadata: {}
Parameters
Inputs
| Parameter | Type | Description |
|---|---|---|
sources | list[str | Path | ByteStream] | File paths or ByteStream objects to convert. Supported formats are PDF, PNG, JPEG, BMP, TIFF, and WebP. Required. |
meta | dict[str, Any] | list[dict[str, Any]] | None | Optional metadata to attach to the documents. A single dictionary applies to all documents. A list must match the number of sources. |
Outputs
| Parameter | Type | Description |
|---|---|---|
documents | list[Document] | Documents that contain the text extracted from the sources. |
Init Parameters
These are the parameters you can configure in Pipeline Builder:
| Parameter | Type | Default | Description |
|---|---|---|---|
server_url | str | Base URL of the MiniCPM-OCR inference server, for example http://localhost:8000. A trailing /v1 is removed. | |
model | str | minicpm-ocr | Model name served by the inference server. Required. |
backend | str | vllm | Server dialect: vllm, llamacpp, ollama, or openai. SGLang and LMDeploy use openai. |
api_key | Secret | Secret.from_env_var("MINICPM_OCR_API_KEY", strict=False) | Optional API key for the inference server. |
layout_model_dir | str | None | None | Directory of the PP-DocLayout model. If unset, the model is stored in ~/.cache/minicpm-ocr/layout and downloaded there on warm_up() if missing. |
layout_device | str | cpu | Device for layout detection, for example cpu or cuda:0. |
request_timeout | float | 300.0 | Timeout in seconds for one recognition request. |
page_workers | int | 1 | Pages rendered and detected concurrently. |
max_workers | int | 1 | Recognition requests in flight concurrently. |
store_full_path | bool | False | If true, stores the full source file path in the document metadata. If false, stores only the file name. |
Run Method Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
sources | list[str | Path | ByteStream] | File paths or ByteStream objects to convert. | |
meta | dict[str, Any] | list[dict[str, Any]] | None | None | Optional metadata to attach to the documents. |
Related Information
Was this page helpful?