OpenSearchBM25Retriever

Retrieve documents from an OpenSearchDocumentStore using the keyword-based BM25 algorithm.

Key Features

• Keyword-based full-text retrieval using BM25 scoring.
• Configurable number of results with top_k.
• Supports fuzzy matching for handling typos and misspellings.
• Supports metadata filtering to narrow down the search space.
• Supports custom OpenSearch queries for advanced use cases.
• Configurable filter policy (replace or merge) for runtime filters.

Configuration

1. Drag the OpenSearchBM25Retriever component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   • Configure the OpenSearchDocumentStore with your OpenSearch instance details.
   • Set top_k to control the maximum number of documents to retrieve.
4. Go to the Advanced tab to configure fuzziness, all_terms_must_match, scale_score, filter_policy, and custom_query.

Connections

OpenSearchBM25Retriever receives the user query as a text string, typically from the Input component. It sends retrieved documents to downstream components such as PromptBuilder or a ranker.

Source Code

To check this component's source code, open bm25_retriever.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  OpenSearchBM25Retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      filters:
      fuzziness: 0
      top_k: 10
      scale_score: false
      all_terms_must_match: false
      filter_policy: replace
      custom_query:
      raise_on_failure: true
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: my-index
          embedding_dim: 768

Using the Component in a Pipeline

# haystack-pipeline
components:
  OpenSearchBM25Retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      filters:
      fuzziness: 0
      top_k: 10
      scale_score: false
      all_terms_must_match: false
      filter_policy: replace
      raise_on_failure: true
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: my-index
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          create_index: true

connections: []

max_runs_per_component: 100

metadata: {}

inputs:
  query:
  - OpenSearchBM25Retriever.query

outputs:
  documents: OpenSearchBM25Retriever.documents

Parameters

Inputs

Parameter	Type	Description
query	str	The text query to search for.
filters	Optional[Dict[str, Any]]	Filters to apply to the search results.
top_k	Optional[int]	The maximum number of documents to return.
fuzziness	Optional[Union[int, str]]	Fuzziness for the BM25 query to allow for typos.
scale_score	Optional[bool]	Whether to scale the document scores.
custom_query	Optional[Dict[str, Any]]	A custom OpenSearch query to override the default BM25 query.
all_terms_must_match	Optional[bool]	Whether all query terms must be present in the matching documents.

Outputs

Parameter	Type	Description
documents	List[Document]	The retrieved documents.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
document_store	OpenSearchDocumentStore		An instance of OpenSearchDocumentStore.
filters	Optional[Dict[str, Any]]	None	Default filters applied when running the retriever.
fuzziness	Union[int, str]	0	Fuzziness for matching. Use "AUTO" for automatic fuzziness based on term length.
top_k	int	10	The maximum number of documents to retrieve.
scale_score	bool	False	Whether to scale scores to the range 0 to 1.
all_terms_must_match	bool	False	If True, all query terms must appear in matching documents.
filter_policy	Union[str, FilterPolicy]	FilterPolicy.REPLACE	Policy for how runtime filters are applied relative to init-time filters.
custom_query	Optional[Dict[str, Any]]	None	A custom OpenSearch query structure.
raise_on_failure	bool	True	Whether to raise an error when a query fails.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
query	str		The text query to search for.
filters	Optional[Dict[str, Any]]	None	Filters to apply at query time.
all_terms_must_match	Optional[bool]	None	Override the init-time setting.
top_k	Optional[int]	None	Override the init-time top_k setting.
fuzziness	Optional[Union[int, str]]	None	Override the init-time fuzziness setting.
scale_score	Optional[bool]	None	Override the init-time scale_score setting.
custom_query	Optional[Dict[str, Any]]	None	Override the init-time custom query.

Related Information

• Pipeline Components