Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Pass Runtime Parameters with the REST API

Override component parameters for a single request through the Search, Chat Stream, and Chat Completions API endpoints.

About This Task​

Runtime parameters override the parameters set in your pipeline configuration for one query. For the parameters you can change and the JSON format to use, see Modify Pipeline Parameters at Query Time.

Prerequisites​

  • You need a Haystack Platform API key. For details, see Generate API Keys.
  • You need a deployed, active pipeline.

Pass Parameters Through the Search API​

When querying your pipeline with the Search API endpoint, include the params field in your request. Inside params, specify the component names and the parameters you want to modify.

This is an example request that changes the embedding_retriever's top_k to 1 for the query "Who was in the first all-girl jazz bands?":

curl --request POST \
     --url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/pipelines/<YOUR_PIPELINE>/search \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "debug": true,
  "params": {
    "embedding_retriever": { "top_k": 1 }
  },
  "view_prompts": false,
  "queries": [
    "who was in the first all-girl jazz bands?"
  ]
}
'

Pass Parameters Through the Chat Stream API​

When querying your pipeline with the Chat Stream API endpoint, include the params field in your request body. This works both for deployed pipelines and for pipelines running in serverless mode (without prior deployment).

This is an example request that limits the retriever to one result:

curl --request POST \
     --url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/pipelines/<YOUR_PIPELINE>/chat-stream \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "query": "Who was in the first all-girl jazz bands?",
  "params": {
    "retriever": { "top_k": 1 }
  },
  "search_session_id": "your-session-id"
}
'

Routing Filters to Components in Serverless Mode​

When running a pipeline in serverless mode via chat-stream, you can also pass Haystack filters at query time. For filters to reach the right component, you must add an inputs section to your pipeline YAML that maps the filters key to the target component input using the format component_name.input_name:

inputs:
  filters:
    - retriever.filters

You can map filters to multiple components at once:

inputs:
  filters:
    - bm25_retriever.filters
    - embedding_retriever.filters

With this mapping in place, filters you pass in the chat-stream request body are forwarded to all listed components:

curl --request POST \
     --url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/pipelines/<YOUR_PIPELINE>/chat-stream \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "query": "What did Alice write?",
  "filters": {
    "field": "meta.category",
    "operator": "==",
    "value": "news"
  },
  "search_session_id": "your-session-id",
  "serverless": true,
  "pipeline_version_id": "your-pipeline-version-id"
}
'
Important

If the inputs.filters mapping in your pipeline YAML is malformed (for example, a list entry is missing the dot separator between component name and input name), the request returns a 400 error. Make sure each entry follows the component_name.input_name format.

Passing Extra OpenAI Parameters Through Chat Completions​

When you query a pipeline or a service through the OpenAI-compatible /chat/completions endpoint, you can send extra OpenAI generation parameters besides messages, model, and stream, for example temperature, top_p, and max_completion_tokens. This lets you tune the LLM's output for a single request without changing the pipeline file. This works for both the pipeline-level and the deployment-level /chat/completions endpoint.

The platform drops extras that your pipeline YAML inputs mapping doesn't name. To forward them, add an inputs section to your pipeline YAML that maps generation_kwargs to the target component's generation_kwargs input:

inputs:
  messages:
    - agent.messages
  generation_kwargs:
    - agent.generation_kwargs

With this mapping in place, extras from a request like this one are merged into the chat generator at run time:

curl --request POST \
     --url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/pipelines/<YOUR_PIPELINE>/chat/completions \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "messages": [{"role": "user", "content": "What is the weather in Berlin?"}],
  "model": "<YOUR_WORKSPACE>/<YOUR_PIPELINE>",
  "temperature": 1.0,
  "max_completion_tokens": 512
}
'

The mapping points to one component. If your pipeline has more than one LLM, the parameters apply only to the component you map them to.

info

Chat Completions doesn't support client-side tool calling. The platform ignores the tools and tool_choice fields in the request. If your pipeline uses an Agent with its own configured tools, the Agent still runs those tools inside the pipeline.

Important

If the inputs mapping in your pipeline YAML is malformed (for example, a list entry is missing the dot separator between component name and input name), the request returns a 400 error. Make sure each entry follows the component_name.input_name format.