Deploy a Pipeline
Learn how to deploy a Haystack pipeline you built locally as a service on Haystack Enterprise Platform.
About This Task
When deploying a pipeline, follow the flow of validating, running, and then deploying. This is to make sure the pipeline is correctly configured and outputs the expected results. You can iterate on your pipeline by editing the local file and re-running the validate, run, and deploy commands. Every time you deploy, it creates a new revision of the service, so you can roll changes out incrementally.
The haystack-enterprise CLI takes your local pipeline through three commands: validate, run, and deploy. All three read the same pipeline file, so what you validate is what you deploy.
Deploying is iterative. Edit your pipeline locally, then validate, run, and deploy it again as many times as you need.
Pipeline File
All commands below take the same first argument: the path to the local Python file that defines your pipeline or agent (pipeline.py in the examples). When a file defines more than one pipeline instance or factory, use --entrypoint to pick one.
Your pipeline loads in your project's Python environment (an auto-detected virtual environment near the file, or the interpreter you pass with --python). The CLI's own environment doesn't need your pipeline's dependencies installed.
Services
You deploy your pipeline as a service on the platform. Service offers a stable endpoint that remains the same even if you make changes to the pipeline. To learn more about services, see Services.
Prerequisites
- Install the SDK.
- Set Up the SDK.
- A local Python file that defines a Haystack pipeline or agent.
- A basic understanding of services.
Deploy a Pipeline
When using the Haystack Enterprise SDK on Windows, if haystack-enterprise isn't on your PATH, replace it with python -m haystack_enterprise_sdk.cli, for example:
haystack-enterprise validate pipeline.py
becomes
python -m haystack_enterprise_sdk.cli validate pipeline.py
Validate your pipeline to check that it's deployable, run it to check the output, then deploy it as a service.
In these steps, pipeline.py is the path to the local Python file that defines your pipeline or agent. Replace it with the path to your own file.
- Validate your pipeline:
haystack-enterprise validate pipeline.py
The first validation may be slow as the platform builds an environment for the version of haystack-ai your pipeline pins.
If the validation is successful, it prints the Pipeline is valid. message. If you see any errors, fix them and run the command again.
- Run your pipeline with a query:
haystack-enterprise run pipeline.py --query "What is deepset?"
- Deploy your pipeline replacing
my-servicewith the name of your service:
# reuse the service (or create it serverless if it doesn't exist), then activate the revision
haystack-enterprise deploy pipeline.py my-service
If the service doesn't exist, the command creates it. When the service starts serving requests, deploy prints its OpenAI-compatible chat-completions endpoint you can use to call the service from your application.
Every deploy creates a revision, and the revision comment is what tells revisions apart in the service's revision history. By default, the comment is the short sha and subject of the latest git commit. If you deploy with uncommitted changes, or from a file that isn't in a git repository, that comment doesn't describe what you deployed. In an interactive terminal, deploy then asks for a comment before it starts, showing the auto-generated one as the default:
Revision comment [2113194 feat(cli): … (dirty)]:
Type a short description of your change, for example Bump embedder to v2, or press Enter to keep the auto-generated comment. To skip the prompt, pass the comment with -m. deploy never prompts when the git tree is clean, when you pass -m, when the input isn't a terminal, or when the CI environment variable is true, so scripts and CI jobs don't wait for input.
Every deployed pipeline needs to know which input receives the query and which output returns the result. If your pipeline uses the conventional socket names, deploy works this out on its own.
The conventional names are:
queryormessagesfor the input.answers,replies, ordocumentsfor the output.
If a name doesn't match, deploy asks you to choose. It asks only about the one it couldn't work out:
Which socket receives the query?
1. greeter.name (str, mandatory)
0. not mapped
> 1
You can list your inputs and outputs in a <pipeline>.io.yaml file, or pass them with --io-config. Then deploy runs without stopping to ask.
If you press Ctrl-C while the service is rolling out, the rollout keeps going on the platform.
haystack-enterprise service-status my-service
What the Exit Code Means
The exit code of deploy tells you whether the new revision is serving, so you can rely on it to pass or fail a CI job:
- Exit 0: The revision is serving. A serverless service activated it, or a managed service reached
DEPLOYEDorIDLE. Exit 0 also means you chose not to wait: you passed--skip-activation, which pushes the revision without activating it, or you interrupted the command with Ctrl-C. - Exit 1: The rollout failed, or it didn't finish within the wait limit of 10 minutes. After a timeout, the rollout continues on the platform and the service may still come up. Check with
haystack-enterprise service-status my-service, then re-run the job or fix the service. For causes and fixes, see Troubleshoot Deployments.
To push a revision from CI without waiting for a rollout, use --skip-activation and activate the revision later.
What a Push Records About Its Source
Every revision you push with the CLI or the Python SDK is stored together with where it came from, so you can trace a running service back to its code. You don't set any flag or option, and a push never fails because this information is missing.
The push records:
- The repository as
host/path(for example,github.com/org/repo), taken from theoriginremote. - The branch, the commit SHA, and whether the working tree has uncommitted changes.
- The path of the pipeline file relative to the repository root.
- Whether the push ran in an automated pipeline or by hand. A push counts as automated when
CIis set to anything other than an empty value,0, orfalse. IfCIisn't set, the CLI also detects Azure Pipelines, Jenkins, TeamCity, AWS CodeBuild, and Bamboo. If your pipeline isn't detected, setCI=true. - Which client sent the push:
Haystack Enterprise CLIorHaystack Enterprise SDK.
The push leaves out anything it can't determine. A detached HEAD has no branch, a new repository has no commit, and a checkout without an origin remote has no repository. Outside a git repository, or when git isn't installed, the push sends only the CI flag and the client name.
The push never collects your hostname, your operating-system user name, the commit author's name or email, absolute paths, or the raw remote URL. It drops credentials embedded in a remote, such as the job token a CI system adds to a GitLab URL, and it drops ports.
Examples
- Validate your pipeline with a specific entrypoint with your own interpreter and mapping:
haystack-enterprise validate pipeline.py \
--entrypoint my_pipeline \
--python .venv/bin/python \
--io-config pipeline.io.yaml
- Validate a specific pipeline when the Python file defines more than one pipeline:
haystack-enterprise validate pipeline.py --entrypoint my_pipeline
- Load a pipeline with a specific Python interpreter
haystack-enterprise validate pipeline.py --python .venv/bin/python
- Validate a pipeline against an explicit input and output mapping:
haystack-enterprise validate pipeline.py --io-config io.yaml
- Deploy to an existing service:
haystack-enterprise deploy pipeline.py my-service
- Deploy to a specific workspace:
haystack-enterprise deploy pipeline.py my-service --workspace-name my-workspace
- Create a provisioned service with explicit sizing:
haystack-enterprise deploy pipeline.py my-service --managed --service-level PRODUCTION --cpu 2
- Add a comment to the revision:
haystack-enterprise deploy pipeline.py my-service -m "My first deployment"
Without -m, the revision comment is the short sha and subject of your latest git commit, so each revision shows what changed. If you have uncommitted changes to tracked files, the comment ends with (dirty). Outside a git repository, the comment names the pipeline file.
- Push a revision without rolling it out:
haystack-enterprise deploy pipeline.py my-service --skip-activation
- Require that a service already exists:
haystack-enterprise deploy pipeline.py my-service --no-create
- Preview the transformed YAML without deploying (no API credentials required):
haystack-enterprise deploy pipeline.py my-service --dry-run --output out.yaml
- Deploy with INFO and DEBUG logs turned on:
haystack-enterprise --verbose deploy pipeline.py my-service
- Check the service status:
haystack-enterprise service-status my-service
What To Do Next
-
Call the deployed service using the endpoint printed by the
deploycommand.POST https://api.cloud.deepset.ai/api/v1/workspaces/<workspace>/deployments/<deployment-id>/chat/completionsSend
{"model": "<workspace>/<service-name>", "messages": [...]}with anAuthorization: BearerAPI key and you get back a server-sent-event stream ofchat.completion.chunkobjects. You can use it with any OpenAI client. Pass the endpoint URL up to and including/deployments/<deployment-id>as thebase_urlof the OpenAI client. -
Share a link to your pipeline prototype for others to try. A shareable prototype link opens a chat UI for your pipeline. The SDK never creates it unless you ask. Pass the following flags to the
deploycommand:# deploy and create a share link that expires in 7 days, no login requiredhaystack-enterprise deploy pipeline.py my-service --share --share-expiration-days 7 --no-share-login-required--sharecreates the link. Because the chat UI routes through the pipeline's input/output mapping, this is also what asks you to review that mapping and offers to save it as<pipeline>.io.yaml.--share-expiration-dayssets how long the link stays valid. The default is 30.--share-login-required/--no-share-login-requiredcontrol whether recipients must log in.
Sharing requires the service to be deployed, so you can't combine
--sharewith--skip-activation. If the platform doesn't classify your pipeline as a chat pipeline, the link is still created but the CLI warns that the chat UI may not render the output correctly. Setpipeline_output_type: chatin the io-config if that classification is wrong.For more information on sharing pipelines, see Share a Pipeline.
-
Iterate on your pipeline by editing the local file and re-running the
validate,run, anddeploycommands. Every time you deploy, it creates a new revision of the service, so you can roll changes out incrementally. -
Run your pipeline in CI by setting the
CIenvironment variable totrue. This disables the progress bars and prints the output as plain text. Setting theCIenvironment variable tofalseforces the animations back on.IfCIisn't set, the CLI also treats Azure Pipelines, Jenkins, TeamCity, AWS CodeBuild, and Bamboo as CI.In a GitHub Actions
pull_requestrun,actions/checkoutchecks out a temporary merge commit, so the default revision comment reads likeMerge 1234abc into 9876fed. To use the pull request title instead, pass it with-m. Read the title from an environment variable rather than inline${{ }}, because the title is user input:- name: Deployenv:PR_TITLE: ${{ github.event.pull_request.title }}run: uv run haystack-enterprise deploy pipeline.py my-service -m "$PR_TITLE"A
pushrun on a squash-merged branch needs no extra step: the subject of HEAD is already the pull request title.
Related Information
Was this page helpful?