Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Connect to a Published ServiceBeta

Beta
This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

Call a published service from your own app, from the built-in chat, from an OpenAI-compatible chat client, or from a Model Context Protocol (MCP) client. The service endpoint stays the same when you publish a new pipeline version.

About This Task​

A service is a stable endpoint that runs the pipeline version you published in Gateway. You authenticate with a workspace API key, or with OAuth and dynamic client registration (DCR) when that option is enabled, and send queries with one of the chat APIs:

  • Chat returns one JSON response when the run finishes. Conversation history comes from a search session.
  • Chat Stream uses the same conversation model and sends the answer as Server-Sent Events (SSE) while it is generated.
  • Chat Completions uses the OpenAI Chat Completions contract, so OpenAI-compatible clients can call the service.

You can also send a query in the built-in chat by sharing a prototype link, or expose the service on the workspace MCP server and call it as a tool.

Prerequisites​

Get the Service Endpoint​

  1. In Haystack Enterprise Platform, click Gateway and open the service you want to call.
  2. Click Connect.
  3. Under API format, select the format you want to use. Select Completions API for OpenAI-compatible clients.
  4. In the Endpoint card, copy the URL. You don't need to update it in your app when you publish a new pipeline version.

Authenticate with a workspace API key as a Bearer token in the Authorization header. To create or manage keys from this page, click Manage API keys. Store API keys in a secrets manager and never expose them in client-side code.

If OAuth with DCR is enabled, a client can sign in with OAuth instead of an API key. The client registers itself and completes the sign-in.

Choose a Chat Endpoint​

Chat​

Use Chat when your app sends a query and waits for the full answer. For conversational services, the platform appends prior turns from a search session. If you omit a search session ID, the platform creates a new session and returns it in the X-Search-Session-Id response header so you can reuse it on the next turn.

For request and response fields, see Chat.

Chat Stream​

Use Chat Stream when your app should show the answer as it is generated. Search sessions group turns into a conversation, matching Chat. The response is an SSE stream of events such as text deltas, tool calls, reasoning, and the final result.

For the event schema and options, see Chat Stream.

Chat Completions​

Use Chat Completions when you connect a tool that expects the OpenAI /chat/completions contract, such as an OpenAI SDK or an OpenAI-compatible chat client. It's a compatibility layer on top of the service. If you build your own app, use Chat Stream or Chat instead.

Chat Completions doesn't use search sessions. Your client sends the conversation history with each request. You identify the service with the model field in the form {workspace_name}/{deployment_id}. To list the model IDs you can call, use List Models. The platform stores each run in search history, so you can trace the query afterward.

Chat Completions doesn't support client-side tool calling: the platform ignores tools and tool_choice. If your service uses an Agent with its own tools, the Agent still runs them inside the service. To forward generation parameters such as temperature, see Passing Extra OpenAI Parameters Through Chat Completions.

For request and response fields, streaming, file references, and error codes, see Chat Completions.

Connect OpenAI-Compatible Clients​

You can point an OpenAI-compatible chat client at Chat Completions and run the service as a hosted model. Thunderbolt and Open WebUI are two examples. Other clients that use the OpenAI Chat Completions API work the same way.

Thunderbolt logo
Thunderbolt
Open WebUI logo
Open WebUI

In the client, set the base URL to the Chat Completions endpoint for your workspace and choose the model ID for the service. Authenticate with a workspace API key, or with OAuth and DCR when the client supports it and DCR is enabled. If the client can fetch models, use List Models so it can discover the services in the workspace. Follow the client's own setup for where those values go. For a walkthrough of Open WebUI, Thunderbolt, and the Vercel AI SDK, see Tutorial: Connecting OpenAI-Compatible Clients to a Service. Chat Completions doesn't support client-side tool calling, so a client can't pass its own tools to the service.

Use the Chat Interface​

You can send a query to a service in the built-in chat, without calling the API yourself.

  1. In Haystack Enterprise Platform, click Gateway and open the service.
  2. On Overview, click Share prototype.
  3. Generate a link and open it. The page is a chat where you, or anyone you send the link to, can query the service.

To set the link expiration, require login, turn on filters, and apply your logo and colors, see Share a Service.

Use the Workspace MCP Server​

You can expose a service as a tool on your workspace MCP server, so MCP-capable AI assistants, such as Claude Desktop or Cursor, can call the service directly. Using a service as an MCP tool is recommended for production because the service endpoint stays stable when you publish a new pipeline version.

To turn on Use as MCP tool for a service, see Manage a Service. To connect an MCP client to the workspace server, see Use Your Pipelines as MCP Tools.

What To Do Next​