PyPDFToDocument

Convert PDF files to documents your pipeline can query. This component uses the PyPDF library to extract text from PDFs and supports both plain and layout-aware extraction modes.

PyPDFToDocument uses the PyPDF library to extract text from PDF files. It supports two extraction modes: plain (default) and layout (experimental, adheres to the rendered layout of the PDF).

Key Features

• Two extraction modes: plain text and layout-aware (experimental).
• Configurable orientation support for rotated pages in plain mode.
• Fine-grained layout mode parameters for vertical spacing and font handling.
• Optional metadata attachment to resulting documents.

Configuration

1. Drag the PyPDFToDocument component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. Configure the component settings:
   • Set the Extraction Mode to choose between plain (default) or layout (experimental layout-aware extraction).
   • Plain mode: set plain_mode_orientations to specify which page orientations to look for, and plain_mode_space_width to control default space width.
   • Layout mode: set layout_mode_space_vertically, layout_mode_scale_weight, layout_mode_strip_rotated, and layout_mode_font_height_weight to tune layout extraction.
   • Set Store Full Path to control whether the full file path or just the file name is stored in document metadata.

Connections

PyPDFToDocument accepts a list of file paths or ByteStream objects through its sources input. It outputs a list of Document objects.

It typically connects with:

• FileTypeRouter: receives PDF files routed by MIME type.
• DocumentJoiner: sends converted documents to join with output from other converters before further processing.

Source Code

To check this component's source code, open pypdf.py in the Haystack repository.

Usage Examples

Basic Configuration

  PyPDFToDocument:
    type: haystack.components.converters.pdf.PyPDFToDocument
    init_parameters:
      extraction_mode: PyPDFExtractionMode.PLAIN
      plain_mode_orientations:
      - 0
      - 90
      - 180
      - 270
      plain_mode_space_width: 200.0
      layout_mode_space_vertically: true
      layout_mode_scale_weight: 1.25
      layout_mode_strip_rotated: true
      layout_mode_font_height_weight: 1.0
      store_full_path: false

Using the Component in an Index

In this index, PyPDFToDocument receives PDF files from FileTypeRouter and sends them to DocumentJoiner.

components:
  FileTypeRouter:
    type: haystack.components.routers.file_type_router.FileTypeRouter
    init_parameters:
      mime_types:
      - text/plain
      - application/pdf
      - text/markdown
  TextFileToDocument:
    type: haystack.components.converters.txt.TextFileToDocument
    init_parameters:
      encoding: utf-8
      store_full_path: false
  PyPDFToDocument:
    type: haystack.components.converters.pdf.PyPDFToDocument
    init_parameters:
      extraction_mode: PyPDFExtractionMode.PLAIN
      plain_mode_orientations:
      - 0
      - 90
      - 180
      - 270
      plain_mode_space_width: 200.0
      layout_mode_space_vertically: true
      layout_mode_scale_weight: 1.25
      layout_mode_strip_rotated: true
      layout_mode_font_height_weight: 1.0
      store_full_path: false
  MarkdownToDocument:
    type: haystack.components.converters.markdown.MarkdownToDocument
    init_parameters:
      table_to_single_line: false
      progress_bar: true
      store_full_path: false
  DocumentJoiner:
    type: haystack.components.joiners.document_joiner.DocumentJoiner
    init_parameters:
      join_mode: concatenate
      weights:
      top_k:
      sort_by_score: true
  DocumentSplitter:
    type: haystack.components.preprocessors.document_splitter.DocumentSplitter
    init_parameters:
      split_by: word
      split_length: 200
      split_overlap: 0
      split_threshold: 0
      splitting_function:
      respect_sentence_boundary: false
      language: en
      use_split_rules: true
      extend_abbreviations: true
  DocumentWriter:
    type: haystack.components.writers.document_writer.DocumentWriter
    init_parameters:
      policy: NONE
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: pdf-index
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          method:
          mappings:
          settings:
          create_index: true
          http_auth:
          use_ssl:
          verify_certs:
          timeout:

connections:
- sender: FileTypeRouter.text/plain
  receiver: TextFileToDocument.sources
- sender: FileTypeRouter.application/pdf
  receiver: PyPDFToDocument.sources
- sender: FileTypeRouter.text/markdown
  receiver: MarkdownToDocument.sources
- sender: TextFileToDocument.documents
  receiver: DocumentJoiner.documents
- sender: PyPDFToDocument.documents
  receiver: DocumentJoiner.documents
- sender: MarkdownToDocument.documents
  receiver: DocumentJoiner.documents
- sender: DocumentJoiner.documents
  receiver: DocumentSplitter.documents
- sender: DocumentSplitter.documents
  receiver: DocumentWriter.documents

max_runs_per_component: 100

Parameters

Inputs

Parameter	Type	Default	Description
sources	List[Union[str, Path, ByteStream]]		List of file paths or ByteStream objects to convert.
meta	Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]	None	Optional metadata to attach to the documents. This value can be a list of dictionaries or a single dictionary. If it's a single dictionary, its content is added to the metadata of all produced documents. If it's a list, its length must match the number of sources, as they are zipped together. For ByteStream objects, their meta is added to the output documents.

Outputs

Parameter	Type	Description
documents	List[Document]	A list of converted documents.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
extraction_mode	Union[str, PyPDFExtractionMode]	PyPDFExtractionMode.PLAIN	The mode to use for extracting text from a PDF. Layout mode is an experimental mode that adheres to the rendered layout of the PDF.
plain_mode_orientations	tuple	(0, 90, 180, 270)	Tuple of orientations to look for when extracting text from a PDF in plain mode. Ignored if extraction_mode is PyPDFExtractionMode.LAYOUT.
plain_mode_space_width	float	200.0	Forces default space width if not extracted from font. Ignored if extraction_mode is PyPDFExtractionMode.LAYOUT.
layout_mode_space_vertically	bool	True	Whether to include blank lines inferred from y distance + font height. Ignored if extraction_mode is PyPDFExtractionMode.PLAIN.
layout_mode_scale_weight	float	1.25	Multiplier for string length when calculating weighted average character width. Ignored if extraction_mode is PyPDFExtractionMode.PLAIN.
layout_mode_strip_rotated	bool	True	Layout mode does not support rotated text. Set to False to include rotated text anyway. If rotated text is discovered, layout will be degraded and a warning will be logged. Ignored if extraction_mode is PyPDFExtractionMode.PLAIN.
layout_mode_font_height_weight	float	1.0	Multiplier for font height when calculating blank line height. Ignored if extraction_mode is PyPDFExtractionMode.PLAIN.
store_full_path	bool	False	If True, the full path of the file is stored in the metadata of the document. If False, only the file name is stored.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
sources	List[Union[str, Path, ByteStream]]		List of file paths or ByteStream objects to convert.
meta	Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]	None	Optional metadata to attach to the documents. This value can be a list of dictionaries or a single dictionary. If it's a single dictionary, its content is added to the metadata of all produced documents. If it's a list, its length must match the number of sources, as they are zipped together. For ByteStream objects, their meta is added to the output documents.