Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Update Pipeline Settings

Change a pipeline's settings to control its hosting, speed it up with a GPU, or pin its Haystack version.


Open Pipeline Settings​

  1. In Haystack Enterprise Platform, go to Pipelines.
  2. Find the pipeline you want to configure and click Settings next to its name.
The Pipeline Settings tab

Configure Pipeline Hosting​

Use the Pipeline Hosting section to control how your pipeline scales and how long it stays active when idle. These settings apply only to pipelines deployed without a service. When a pipeline is published to a service, the service's hosting settings take over.

For the difference between pipeline and service hosting settings, see Gateway.

For production and business-critical pipelines, we recommend at least 2 replicas always available and a longer idle timeout, like 1 day or more. For test pipelines or pipelines with low traffic, you can keep 1 replica at a minimum and a shorter idle timeout to save resources. For details, see Pipeline Hosting Settings.

Set the Production Service Level​

Turn on Production to apply the recommended production hosting values automatically:

  • Minimum replicas: 2
  • Idle timeout: 30 days

Turn it off to switch to development defaults:

  • Minimum replicas: 0
  • Idle timeout: 20 minutes

You can also set the values manually at any time, regardless of whether Production is on or off. Adding more replicas increases your costs.

Enable GPU Acceleration​

By default, pipelines run on CPU. You can enable GPU acceleration for a pipeline to speed it up. This is especially useful for pipelines that include components that run faster on a GPU, like custom components that use AI models.

When you enable GPU acceleration, the pipeline checks whether a component needs a GPU and assigns one automatically. GPUs are only used when required and are not reserved for the entire pipeline run.

If GPU acceleration is disabled and your pipeline includes components that rely on a GPU, those components run on the CPU instead. This can slow down processing and may cause timeouts, especially for larger or more complex pipelines.

Turn on GPU Enabled to allocate a GPU to your pipeline.

Pin the Haystack Version​

Haystack Enterprise Platform uses Haystack Open Source as its underlying framework. You can choose which version of Haystack your pipeline runs on to keep behavior consistent and avoid breaking changes.

Choose a version from the Haystack Version list to pin the version your pipeline runs on. The selected version is saved to the pipeline configuration as a haystack-ai dependency, so your pipeline always uses that version. If you leave it unset, the pipeline runs on the default supported version.

You must pin a Haystack version before you can publish a pipeline version to a service. Pin a version in these settings first. The pin can be the Haystack Version setting or a haystack-ai==<version> entry under dependencies in the pipeline YAML. REST API requests and SDK or CLI pushes need the pin too. For details, see Publish a Pipeline to a Service.

Configure History Granularity​

History granularity controls how much of the conversation history gets sent to the pipeline on each turn. Use this setting for chat-based pipelines.

Choose one of the following from the Granularity dropdown:

  • Query & answer: Sends only the user query and final answer from each previous turn. Use this to keep context lightweight.
  • All messages: Sends every message from previous turns, including tool calls and reasoning. Use this when the pipeline needs full context to respond accurately.

Enable Session Storage​

Turn on Session storage to give each session an isolated file system. This lets tools like bash store and reuse files across requests within the same session.

When you turn on session storage, the platform automatically treats the pipeline as a chat output type in Playground and the API, even if the pipeline has no generators or prompts configured. To use a different output type instead, set pipeline_output_type explicitly in the pipeline YAML. For details, see Set Pipeline Output Type.

A session keeps one saved snapshot of its files. If two runs on the same session ID overlap and session snapshots are stored in S3, a save fails when another run has replaced the snapshot since this run last restored or saved it. The platform keeps the other run's files instead of overwriting them. The later run fails at its final save, while conflicts during periodic saves are logged and the run continues. After a failed final save, requests for that session ID that reach the same pod can fail with a 503 error until the session recovers. To avoid this, send the next request for a session only after the previous run has finished.

View Linked Services​

Check the services where this pipeline is currently published. GPU settings, replicas, and idle timeout for these services are configured per service, not here.

Manage Feedback Tags​

Add tags in the Feedback tags section that users can select when leaving feedback on the pipeline's responses. Tags help you group and filter the feedback you receive.

To add a tag, click + New Tag, type the tag name, and press Enter. To remove a tag, click × next to it.

For details on working with feedback, see Collect Feedback.