OpenSearchEmbeddingRetriever

Retrieve documents from an OpenSearchDocumentStore using vector similarity search.

Key Features

• Dense vector-based retrieval from OpenSearch using k-NN similarity.
• Configurable number of results with top_k.
• Supports metadata filtering to narrow down the search space.
• Supports custom OpenSearch queries for advanced use cases.
• Configurable filter policy (replace or merge) for runtime filters.
• Efficient filtering mode for improved performance with large datasets.

Configuration

1. Drag the OpenSearchEmbeddingRetriever component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   • Configure the OpenSearchDocumentStore with your OpenSearch instance details.
   • Set top_k to control the maximum number of documents to retrieve.
4. Go to the Advanced tab to configure filter_policy, efficient_filtering, and custom_query.

Connections

OpenSearchEmbeddingRetriever receives query embeddings from a text embedder. It sends retrieved documents to downstream components such as PromptBuilder or a ranker.

Source Code

To check this component's source code, open embedding_retriever.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  OpenSearchEmbeddingRetriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      filters:
      top_k: 10
      filter_policy: replace
      custom_query:
      raise_on_failure: true
      efficient_filtering: false
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: my-index
          embedding_dim: 768

Using the Component in a Pipeline

# haystack-pipeline
components:
  OpenSearchEmbeddingRetriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      filters:
      top_k: 10
      filter_policy: replace
      raise_on_failure: true
      efficient_filtering: false
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: my-index
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          create_index: true

connections: []

max_runs_per_component: 100

metadata: {}

inputs:
  query_embedding:
  - OpenSearchEmbeddingRetriever.query_embedding

outputs:
  documents: OpenSearchEmbeddingRetriever.documents

Parameters

Inputs

Parameter	Type	Description
query_embedding	List[float]	The embedding of the query.
filters	Optional[Dict[str, Any]]	Filters to apply to the search results.
top_k	Optional[int]	The maximum number of documents to return.
custom_query	Optional[Dict[str, Any]]	A custom OpenSearch query to use for retrieval.
efficient_filtering	Optional[bool]	Whether to use efficient filtering mode.

Outputs

Parameter	Type	Description
documents	List[Document]	The retrieved documents.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
document_store	OpenSearchDocumentStore		An instance of OpenSearchDocumentStore.
filters	Optional[Dict[str, Any]]	None	Default filters applied when running the retriever.
top_k	int	10	The maximum number of documents to retrieve.
filter_policy	Union[str, FilterPolicy]	FilterPolicy.REPLACE	Policy for how runtime filters are applied relative to init-time filters.
custom_query	Optional[Dict[str, Any]]	None	A custom OpenSearch query structure.
raise_on_failure	bool	True	Whether to raise an error when a query fails.
efficient_filtering	bool	False	When True, uses OpenSearch's post-filter for better performance with large indices.
search_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for the OpenSearch k-NN search request.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
query_embedding	List[float]		The embedding of the query.
filters	Optional[Dict[str, Any]]	None	Filters to apply at query time.
top_k	Optional[int]	None	Override the init-time top_k setting.
custom_query	Optional[Dict[str, Any]]	None	Override the init-time custom query.
efficient_filtering	Optional[bool]	None	Override the init-time efficient_filtering setting.

Related Information

• Pipeline Components