Move Sessions, Feedback, Metadata, and Health Checks to a Service with the REST APIBeta

Beta

This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

Use this guide after you publish a pipeline to a service, when your app still calls pipeline endpoints and must switch to the service's endpoints.

About This Task

When you migrate a pipeline to a service, the app that calls the pipeline must switch to the service endpoint. Calls for search and chat change their URL. For sessions, feedback, metadata, and health status, the change depends on the feature:

Feature	What changes
Search and chat	Call the service instead of the pipeline.
Deleting sessions	Nothing. The endpoint belongs to the workspace and works for old and new sessions.
Creating sessions	Create sessions on the service.
Feedback	Send feedback to the service instead of the pipeline.
Search metadata	The service has no metadata endpoint. Keep reading metadata from the pipeline and add the pipeline_version_id parameter.
Health status	Read the status of the service.

Replace these placeholders in the examples:

• <WORKSPACE_NAME>: The name of the workspace.
• <DEPLOYMENT_ID>: The ID of the service. In the API, a service is a deployment.
• <SEARCH_SESSION_ID>: The ID of a session.
• <YOUR_API_KEY>: Your Haystack Platform API key.

Prerequisites

• You must have a pipeline published to a service. For details, see Publish a Pipeline to a Service.
• You need an API key. For details, see Generate API Keys.
• Your API key needs read access to services to call the service. Feedback endpoints need the feedback permissions described in Send Feedback to the Service.

Call the Service Instead of the Pipeline

Service endpoints use this base URL:

https://api.cloud.deepset.ai/api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>

To switch a call, replace .../pipelines/<PIPELINE_NAME> in the URL with .../deployments/<DEPLOYMENT_ID>:

• Search: POST .../deployments/<DEPLOYMENT_ID>/search
• Chat with streaming: POST .../deployments/<DEPLOYMENT_ID>/chat-stream

For example, a chat call to a service:

curl --no-buffer --request POST \
  --url 'https://api.cloud.deepset.ai/api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>/chat-stream' \
  --header 'Authorization: Bearer <YOUR_API_KEY>' \
  --header 'Content-Type: application/json' \
  --header 'Accept: text/event-stream' \
  --data '{"query": "What is a servitude?", "include_result": true}'

Keep in mind these differences from the pipeline endpoint:

• Remove serverless and pipeline_version_id from the request. A service always runs its active pipeline version.
• The response format stays the same: delta events, then a result event, then a done event. The delta and result events have an additional stream_id field, and extra_outputs has an additional fallback_info field.
• The query_id in the done event is a request ID that the platform generates. It isn't the ID of the answer. Take the ID of the answer from the result or delta events.
• To receive documents and references, set include_result to true. Its default is false, and without it the stream of /search-stream and /chat-stream contains no result.
• Set client timeouts to at least 300 seconds. A long question can take two to three minutes.

Create and Continue Sessions

A session groups the questions and answers of one conversation. A session belongs to the service it was created for. You can't use it with another service or a pipeline, so sessions from the old pipeline endpoint can't continue on the service.

To create a session before the first question, for example when a user opens a new chat:

curl --request POST \
  --url 'https://api.cloud.deepset.ai/api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>/search_sessions' \
  --header 'Authorization: Bearer <YOUR_API_KEY>'

The request needs no body and returns 201 Created with the session ID:

{"search_session_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}

Send this ID as search_session_id with every question of the conversation. The service uses the previous turns of the session as context. By default, it uses the last three turns. You can change this with chat_history_limit.

Alternatively, send the first question without search_session_id. The service creates a session and returns its ID in the x-search-session-id response header. Send it as search_session_id in the follow-up questions.

Delete Sessions

The endpoint for deleting sessions stays the same:

curl --request DELETE \
  --url 'https://api.cloud.deepset.ai/api/v1/workspaces/<WORKSPACE_NAME>/search_sessions/<SEARCH_SESSION_ID>' \
  --header 'Authorization: Bearer <YOUR_API_KEY>'

• The endpoint belongs to the workspace, not to a pipeline or a service. It deletes sessions you created through the pipeline and new sessions of services.
• Deleting a session also deletes its questions and answers.
• The endpoint returns 204 on success and 404 if the session doesn't exist.
• We don't delete sessions or their history automatically. If you run a job that deletes conversations older than a number of days, keep running it.

Select Sessions to Delete

If your job finds sessions with GET /api/v1/workspaces/<WORKSPACE_NAME>/search_sessions and filters by pipeline name, it can miss the sessions of a service. These sessions belong to the service, not directly to the pipeline. Each session in the response has these fields you can use to select sessions instead:

• deployment_id: The ID of the service the session belongs to.
• search_history.created_at: The time of the first question of the session. A session without questions has no search_history.

Send Feedback to the Service

Send feedback to the service instead of the pipeline:

POST   /api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>/feedback
GET    /api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>/feedback
PATCH  /api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>/feedback/<FEEDBACK_ID>
DELETE /api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>/feedback/<FEEDBACK_ID>

• The request body is the same as for pipeline feedback. result_id and query_id are required, and you can add score, comment, and other fields. Both IDs are in the response of the service.
• Feedback is only possible for runs that were stored in search history, so send the run with store_search_history set to true.
• The list endpoint returns only feedback on the answers of this service. It doesn't include feedback on the pipeline's own runs.
• Creating and updating feedback needs write access to feedback. Listing and deleting feedback needs read and write access to feedback management, respectively.

Read Search Metadata

A service has no metadata endpoint of its own. Keep using the pipeline endpoints and pass the pipeline version that the service currently runs:

GET /api/v1/workspaces/<WORKSPACE_NAME>/pipelines/<PIPELINE_NAME>/meta?pipeline_version_id=<VERSION_ID>
GET /api/v1/workspaces/<WORKSPACE_NAME>/pipelines/<PIPELINE_NAME>/meta/<FIELD_NAME>?pipeline_version_id=<VERSION_ID>
GET /api/v1/workspaces/<WORKSPACE_NAME>/pipelines/<PIPELINE_NAME>/meta/<FIELD_NAME>/aggregation?pipeline_version_id=<VERSION_ID>

1. Get the service:

   curl --request GET \
     --url 'https://api.cloud.deepset.ai/api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>' \
     --header 'Authorization: Bearer <YOUR_API_KEY>'

2. In the response, read pipeline_name for <PIPELINE_NAME> and active_revision.source_version_id for <VERSION_ID>.

3. Call a metadata endpoint with both values.

Always pass pipeline_version_id. If you omit it and the pipeline itself isn't deployed, the request succeeds but returns {}. If you get an empty response, add the parameter with the current source_version_id.

These values change with every new version of the service. Read source_version_id again after you publish a new version, and don't store it long term.

Monitor the Health of a Service

Read the status of a service with the same endpoint you used to get the pipeline name:

GET /api/v1/workspaces/<WORKSPACE_NAME>/deployments/<DEPLOYMENT_ID>

These fields describe the health:

Field	Values	Meaning
status	DEPLOYED, IDLE, DEPLOYMENT_IN_PROGRESS, DEPLOYMENT_FAILED, UNDEPLOYED	The current state of the service.
desired_status	DEPLOYED, UNDEPLOYED	The state the service should reach.
active_revision.status	ACTIVE or another value	ACTIVE means the active version is running. Alert on any other value.

IDLE isn't an error. A service shuts down after 20 minutes without requests and starts again with the next request. That first request takes longer.

We recommend that you alert when:

• status is DEPLOYMENT_FAILED or UNDEPLOYED.
• desired_status isn't DEPLOYED.
• active_revision.status isn't ACTIVE.

The status has limits:

• It doesn't tell you whether the next start of an idle service succeeds.
• In rare cases, DEPLOYED can be a stale value.

Also monitor real requests for 5xx responses, timeouts, and error events in the stream. To check the service actively, send a test request at regular intervals. Each test request wakes up an idle service and uses compute time.

Handle Errors

Code	Meaning	What to do
400	The input mapping or the chat history granularity is invalid.	Fix the request.
401 or 403	The API key is missing or invalid, or it lacks permissions.	Check the key and its permissions.
404	The service isn't found.	Check the service ID.
408	The request took too long.	Retry or narrow the question.
409	The service isn't deployed or can't run.	Check the status of the service.
422	The request validation failed, for example queries is missing on /search.	Fix the request body.
429	You reached the rate limit.	Wait and retry.
500	The pipeline failed.	Retry once, then report the time and the query_id.
503	The service is temporarily unavailable.	Retry after about 30 seconds.

What To Do Next

• Send a test search or chat request to the service and confirm it returns a result. Then check that status is DEPLOYED or IDLE, and update your callers and monitoring to use the new endpoints.

Related Information

• Migrate to Service
• Choose a Chat Endpoint
• Connect to a Published Service
• Monitor a Service
• Manage a Service