Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Renumber split_id in an Index

Number split_id per file so each file appears once on the index's Files tab.

About This Task​

The Files tab lists the document with split_id 0 of each file. If your index gives several documents of a file split_id 0, or no split_id, the file appears once for each of them. The file is still indexed once. For details, see Why a File Appears More Than Once.

To fix it, add a code component that numbers split_id 0, 1, 2, and so on across all documents of a file.

Renumber split_id​

  1. Open your index in Pipeline Builder and add this code component to its YAML, right before the embedder:

    SplitIdRenumber:
      type: deepset_cloud_custom_nodes.code.code_component.Code
      init_parameters:
        code: |
          from dataclasses import replace
          from typing import Dict, List
          from haystack import component, Document


          @component
          class SplitIdRenumber:
              """Numbers split_id 0..n per file, so each file shows up once in the file list."""

              @component.output_types(documents=List[Document])
              def run(self, documents: List[Document]) -> dict:
                  counters: Dict[str, int] = {}
                  result = []
                  for doc in documents:
                      key = str(doc.meta.get("file_id") or doc.meta.get("file_name"))
                      idx = counters.get(key, 0)
                      counters[key] = idx + 1
                      result.append(replace(doc, meta={**doc.meta, "split_id": idx}))
                  return {"documents": result}

  2. Connect it between the component that sends documents to the embedder and the embedder. For example, replace this connection:

    - sender: DocumentSplitter.documents
      receiver: Embedder.documents

    with these two:

    - sender: DocumentSplitter.documents
      receiver: SplitIdRenumber.documents
    - sender: SplitIdRenumber.documents
      receiver: Embedder.documents

  3. Deploy the index and reindex the affected files. Files indexed before the change keep their extra rows until you reindex them. When indexing finishes, each reindexed file appears once on the Files tab.

The component changes only split_id; everything else on each document stays the same.

What To Do Next​