Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

MiniCPMOCRDocumentConverter

Extract text from PDF files and images using a MiniCPM-OCR inference server.

MiniCPMOCRDocumentConverter detects the layout of each page in your pipeline with a PP-DocLayout model, sends the cropped blocks to a MiniCPM-OCR inference server that you host, and returns the recognized text as Markdown in Haystack Document objects. It returns one Document per source, with pages separated by form feed characters (\f). Use it in indexing pipelines to make scanned documents and images searchable.

Key Features​

  • Extracts text from PDF, PNG, JPEG, BMP, TIFF, and WebP files.
  • Attaches optional metadata to the resulting documents. You can pass one metadata dictionary for all documents or a list with one dictionary per source.

Configuration​

  1. Make sure you have a running MiniCPM-OCR inference server that your pipeline can reach, and note its base URL.
  2. Configure MiniCPMOCRDocumentConverter in your pipeline YAML. Set server_url to the base URL of your server and model to the model name your server serves. If your server isn't vLLM, set backend to llamacpp, ollama, or openai.
  3. Run the indexing pipeline on a test file and check that the resulting documents contain the recognized text. If the run fails, check that the server is running and that the pipeline can reach server_url.

The integration is published as the minicpm-ocr-haystack package. It's already available in the runtime, so you don't need to add it to the dependencies block of your pipeline YAML.

Connections​

MiniCPMOCRDocumentConverter accepts file paths or ByteStream objects as sources, and optional metadata as meta. It outputs a list of Document objects (documents).

Connect the pipeline's file input to the sources input. Connect the documents output to DocumentSplitter or DocumentWriter for further processing.

Source Code​

To check this component's source code, browse the minicpm_ocr integration in the Haystack Core Integrations repository.

Usage Examples​

Basic Configuration​

minicpm_ocr_converter:
type: haystack_integrations.components.converters.minicpm_ocr.converter.MiniCPMOCRDocumentConverter
init_parameters:
server_url: http://your-minicpm-ocr-server:8000
model: minicpm-ocr
backend: vllm

Using the Component in a Pipeline​

This indexing pipeline converts uploaded files with OCR, splits the text, and writes the chunks to a document store.

# haystack-pipeline
components:
minicpm_ocr_converter:
type: haystack_integrations.components.converters.minicpm_ocr.converter.MiniCPMOCRDocumentConverter
init_parameters:
server_url: http://your-minicpm-ocr-server:8000
model: minicpm-ocr
backend: vllm
splitter:
type: haystack.components.preprocessors.document_splitter.DocumentSplitter
init_parameters:
split_by: word
split_length: 250
split_overlap: 30
writer:
type: haystack.components.writers.document_writer.DocumentWriter
init_parameters:
document_store:
type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
init_parameters:
hosts:
- ${OPENSEARCH_HOST}
index: minicpm-ocr-demo
embedding_dim: 768
create_index: true

connections:
- sender: minicpm_ocr_converter.documents
receiver: splitter.documents
- sender: splitter.documents
receiver: writer.documents

inputs:
files:
- minicpm_ocr_converter.sources

max_runs_per_component: 100

metadata: {}

Parameters​

Inputs​

ParameterTypeDescription
sourceslist[str | Path | ByteStream]File paths or ByteStream objects to convert. Supported formats are PDF, PNG, JPEG, BMP, TIFF, and WebP. Required.
metadict[str, Any] | list[dict[str, Any]] | NoneOptional metadata to attach to the documents. A single dictionary applies to all documents. A list must match the number of sources.

Outputs​

ParameterTypeDescription
documentslist[Document]Documents that contain the text extracted from the sources.

Init Parameters​

These are the parameters you can configure in Pipeline Builder:

ParameterTypeDefaultDescription
server_urlstrBase URL of the MiniCPM-OCR inference server, for example http://localhost:8000. A trailing /v1 is removed.
modelstrminicpm-ocrModel name served by the inference server. Required.
backendstrvllmServer dialect: vllm, llamacpp, ollama, or openai. SGLang and LMDeploy use openai.
api_keySecretSecret.from_env_var("MINICPM_OCR_API_KEY", strict=False)Optional API key for the inference server.
layout_model_dirstr | NoneNoneDirectory of the PP-DocLayout model. If unset, the model is stored in ~/.cache/minicpm-ocr/layout and downloaded there on warm_up() if missing.
layout_devicestrcpuDevice for layout detection, for example cpu or cuda:0.
request_timeoutfloat300.0Timeout in seconds for one recognition request.
page_workersint1Pages rendered and detected concurrently.
max_workersint1Recognition requests in flight concurrently.
store_full_pathboolFalseIf true, stores the full source file path in the document metadata. If false, stores only the file name.

Run Method Parameters​

ParameterTypeDefaultDescription
sourceslist[str | Path | ByteStream]File paths or ByteStream objects to convert.
metadict[str, Any] | list[dict[str, Any]] | NoneNoneOptional metadata to attach to the documents.