Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Deploy a Pipeline

Learn how to deploy a Haystack pipeline you built locally as a service on Haystack Enterprise Platform.

About This Task​

When deploying a pipeline, follow the flow of validating, running, and then deploying. This is to make sure the pipeline is correctly configured and outputs the expected results. You can iterate on your pipeline by editing the local file and re-running the validate, run, and deploy commands. Every time you deploy, it creates a new revision of the service, so you can roll changes out incrementally.

The haystack-enterprise CLI takes your local pipeline through three commands: validate, run, and deploy. All three read the same pipeline file, so what you validate is what you deploy.

Deploying is iterative. Edit your pipeline locally, then validate, run, and deploy it again as many times as you need.

Pipeline File​

All commands below take the same first argument: the path to the local Python file that defines your pipeline or agent (pipeline.py in the examples). When a file defines more than one pipeline instance or factory, use --entrypoint to pick one.

info

Your pipeline loads in your project's Python environment (an auto-detected virtual environment near the file, or the interpreter you pass with --python). The CLI's own environment doesn't need your pipeline's dependencies installed.

Services​

You deploy your pipeline as a service on the platform. Service offers a stable endpoint that remains the same even if you make changes to the pipeline. To learn more about services, see Services.

Prerequisites​

Deploy a Pipeline​

info

When using the Haystack Enterprise SDK on Windows, if haystack-enterprise isn't on your PATH, replace it with python -m haystack_enterprise_sdk.cli, for example:

haystack-enterprise validate pipeline.py

becomes

python -m haystack_enterprise_sdk.cli validate pipeline.py

Validate your pipeline to check that it's deployable, run it to check the output, then deploy it as a service.

In these steps, pipeline.py is the path to the local Python file that defines your pipeline or agent. Replace it with the path to your own file.

  1. Validate your pipeline:
   haystack-enterprise validate pipeline.py

The first validation may be slow as the platform builds an environment for the version of haystack-ai your pipeline pins.

If the validation is successful, it prints the Pipeline is valid. message. If you see any errors, fix them and run the command again.

  1. Run your pipeline with a query:
haystack-enterprise run pipeline.py --query "What is deepset?"
  1. Deploy your pipeline replacing my-service with the name of your service:
# reuse the service (or create it serverless if it doesn't exist), then activate the revision
haystack-enterprise deploy pipeline.py my-service

If the service doesn't exist, the command creates it. When the service starts serving requests, deploy prints its OpenAI-compatible chat-completions endpoint you can use to call the service from your application.

Every deployed pipeline needs to know which input receives the query and which output returns the result. If your pipeline uses the conventional socket names, deploy works this out on its own.

The conventional names are:

  • query or messages for the input.
  • answers, replies, or documents for the output.

If a name doesn't match, deploy asks you to choose. It asks only about the one it couldn't work out:

Which socket receives the query?
  1. greeter.name (str, mandatory)
  0. not mapped
> 1

You can list your inputs and outputs in a <pipeline>.io.yaml file, or pass them with --io-config. Then deploy runs without stopping to ask.

If you press Ctrl-C while the service is rolling out, the rollout keeps going on the platform. To check the progress, run:

haystack-enterprise service-status my-service

Examples​

  • Validate your pipeline with a specific entrypoint with your own interpreter and mapping:
   haystack-enterprise validate pipeline.py \
     --entrypoint my_pipeline \
     --python .venv/bin/python \
     --io-config pipeline.io.yaml
  • Validate a specific pipeline when the Python file defines more than one pipeline:
haystack-enterprise validate pipeline.py --entrypoint my_pipeline
  • Load a pipeline with a specific Python interpreter
haystack-enterprise validate pipeline.py --python .venv/bin/python
  • Validate a pipeline against an explicit input and output mapping:
haystack-enterprise validate pipeline.py --io-config io.yaml
  • Deploy to an existing service:
haystack-enterprise deploy pipeline.py my-service
  • Deploy to a specific workspace:
haystack-enterprise deploy pipeline.py my-service --workspace-name my-workspace
  • Create a provisioned service with explicit sizing:
haystack-enterprise deploy pipeline.py my-service --managed --service-level PRODUCTION --cpu 2
  • Add a comment to the revision:
haystack-enterprise deploy pipeline.py my-service -m "My first deployment"
  • Push a revision without rolling it out:
haystack-enterprise deploy pipeline.py my-service --skip-activation
  • Require that a service already exists:
haystack-enterprise deploy pipeline.py my-service --no-create
  • Preview the transformed YAML without deploying (no API credentials required):
haystack-enterprise deploy pipeline.py my-service --dry-run --output out.yaml
  • Deploy with INFO and DEBUG logs turned on:
haystack-enterprise --verbose deploy pipeline.py my-service
  • Check the service status:
haystack-enterprise service-status my-service

What To Do Next​

  • Call the deployed service using the endpoint printed by the deploy command.

    POST https://api.cloud.deepset.ai/api/v1/workspaces/<workspace>/deployments/<deployment-id>/chat/completions

    Send {"model": "<workspace>/<service-name>", "messages": [...]} with an Authorization: Bearer API key and you get back a server-sent-event stream of chat.completion.chunk objects. You can use it with any OpenAI client. Pass the endpoint URL up to and including /deployments/<deployment-id> as the base_url of the OpenAI client.

  • Share a link to your pipeline prototype for others to try. A shareable prototype link opens a chat UI for your pipeline. The SDK never creates it unless you ask. Pass the following flags to the deploy command:

    # deploy and create a share link that expires in 7 days, no login required
    haystack-enterprise deploy pipeline.py my-service --share --share-expiration-days 7 --no-share-login-required
    • --share creates the link. Because the chat UI routes through the pipeline's input/output mapping, this is also what asks you to review that mapping and offers to save it as <pipeline>.io.yaml.
    • --share-expiration-days sets how long the link stays valid. The default is 30.
    • --share-login-required / --no-share-login-required control whether recipients must log in.

    Sharing requires the service to be deployed, so you can't combine --share with --skip-activation. If the platform doesn't classify your pipeline as a chat pipeline, the link is still created but the CLI warns that the chat UI may not render the output correctly. Set pipeline_output_type: chat in the io-config if that classification is wrong.

    For more information on sharing pipelines, see Share a Pipeline.

  • Iterate on your pipeline by editing the local file and re-running the validate, run, and deploy commands. Every time you deploy, it creates a new revision of the service, so you can roll changes out incrementally.

  • Run your pipeline in CI by setting the CI environment variable to true. This disables the progress bars and prints the output as plain text. Setting the CI environment variable to false forces the animations back on.