Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Connect to a Published ServiceBeta

Beta
This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

Call a published service from your own app, from the built-in chat, from an OpenAI-compatible chat client, or from a Model Context Protocol (MCP) client. The service endpoint stays the same when you publish a new pipeline version.

About This Task​

A service is a stable endpoint that runs the pipeline version you published in Gateway. You authenticate with a workspace API key, or with OAuth and dynamic client registration (DCR) when that option is enabled, and send queries with one of the chat APIs:

  • Chat returns one JSON response when the run finishes. Conversation history comes from a search session.
  • Chat Stream uses the same conversation model and sends the answer as Server-Sent Events (SSE) while it is generated.
  • Chat Completions uses the OpenAI Chat Completions contract, so OpenAI-compatible clients can call the service.

You can also send a query in the built-in chat by sharing a prototype link, expose the service on the workspace MCP server and call it as a tool, or connect a Slack bot so people can ask the service questions in Slack. For Slack setup steps, see Connect a Service to Slack.

Prerequisites​

Get the Service Endpoint​

  1. In Haystack Enterprise Platform, click Gateway and open the service you want to call.
  2. Click Connect.
  3. Under API format, select the format you want to use. Select Completions API for OpenAI-compatible clients.
  4. In the Endpoint card, copy the URL. You don't need to update it in your app when you publish a new pipeline version.

Authenticate with a workspace API key as a Bearer token in the Authorization header. To create or manage keys from this page, click Manage API keys. Store API keys in a secrets manager and never expose them in client-side code.

If OAuth with DCR is enabled, a client can sign in with OAuth instead of an API key. The client registers itself and completes the sign-in.

Test the Service in the Chat Panel​

To check how a service answers before you wire it into an app, send it messages from the Test this service panel on the Connect tab. The panel sends your messages to the live service, so each test counts toward your usage.

  1. In Haystack Enterprise Platform, click Gateway and open the service.
  2. Click Connect, and open the Test this service panel.
  3. Choose a test mode:
    • Chat lets you hold a conversation with the service. Type a message in the Message this service field and send it. You can also click one of the example messages.
    • API request sends the request for the API format you selected under API format, so you can see what your app would send and receive.
  4. Send a message. To interrupt a reply that is still being generated, click Stop. When the reply arrives, copy it, or click Details to open the response inspector and see how the service produced it.

The panel uses the API format you selected on the Connect tab, so you test the same endpoint your app will call. This affects how follow-up messages behave:

  • With Chat Stream, follow-up messages continue the same session.
  • With Completions API, your app keeps the conversation, so every message sends the full history. There is no chat history to reopen, because the service doesn't store the conversation.

The panel shows which revision the service is running. If a new revision is rolling out, a test can reach either the current or the new revision until the rollout finishes.

Manage Chats and Attach Files​

  • To start over, click New chat.
  • To continue an earlier conversation, open Recent chats and select a chat. A conversation that is too long to reopen in the panel is available in the service history.
  • To send a file along with a message, click Attach files, or drop files into the panel. The service must accept files, and you need permission to upload files to the workspace. With Completions API, files travel inside the request, so each file can be up to 5 MB. Larger files aren't attached.

To send the service query to someone without platform access, use the Test & share card on Overview and click Share a test link. For details, see Use the Chat Interface.

Choose a Chat Endpoint​

To pick between the Chat, Chat Stream, and Chat Completions endpoints, see Choose a Chat Endpoint.

Record Where a Pushed Revision Came From​

When you push a revision from code, you can attach an optional source_metadata object to the request, so that anyone reviewing the revision later can see which commit and file it came from. For the request that pushes a revision, see the REST API reference for revisions. The platform stores the object next to the revision and returns it as source_metadata in the revision response.

The client reports these values and the platform doesn't verify them. All fields are optional, and the platform ignores any other top-level keys:

  • git: The origin in version control. It holds repo (normalized, for example github.com/org/repo, up to 300 characters), branch (up to 255), sha (up to 64), and dirty, which is true when the working tree had uncommitted changes.
  • file: The repository-relative path of the source file, up to 1,024 characters.
  • ci: True when an automated pipeline made the push.
  • custom: A free-form JSON object for your own metadata, up to 8,192 characters when serialized. It can't contain NaN values.

The platform rejects a request when a value exceeds its limit.

The source information doesn't change the revision's config hash, so pushing the same configuration with different source information doesn't create a different configuration. It also never reaches the running service.

Connect OpenAI-Compatible Clients​

You can point an OpenAI-compatible chat client at Chat Completions and run the service as a hosted model. Thunderbolt and Open WebUI are two examples. Other clients that use the OpenAI Chat Completions API work the same way.

Thunderbolt logo
Thunderbolt
Open WebUI logo
Open WebUI

In the client, set the base URL to the Chat Completions endpoint for your workspace and choose the model ID for the service. Authenticate with a workspace API key, or with OAuth and DCR when the client supports it and DCR is enabled. If the client can fetch models, use List Models so it can discover the services in the workspace. Follow the client's own setup for where those values go. For a walkthrough of Open WebUI, Thunderbolt, and the Vercel AI SDK, see Tutorial: Connecting OpenAI-Compatible Clients to a Service. Chat Completions doesn't support client-side tool calling, so a client can't pass its own tools to the service.

Use the Chat Interface​

You can send a query to a service in the built-in chat, without calling the API yourself.

  1. In Haystack Enterprise Platform, click Gateway and open the service.
  2. On Overview, click Share prototype.
  3. Generate a link and open it. The page is a chat where you, or anyone you send the link to, can query the service. If you can't generate a link, check that you have permission to share services in the workspace.

On Overview, the Test & share card also has a Share a test link action for the same link, so people without platform access can try the service.

To set the link expiration, require login, turn on filters, and apply your logo and colors, see Share a Service.

To create links that run the same service with different settings, for example one link per document collection, add default runtime parameters to each link. For details, see Set Default Runtime Parameters for a Service Link.

Use the Workspace MCP Server​

You can expose a service as a tool on your workspace MCP server, so MCP-capable AI assistants, such as Claude Desktop or Cursor, can call the service directly. Using a service as an MCP tool is recommended for production because the service endpoint stays stable when you publish a new pipeline version.

To turn on a service as a tool:

  1. In Haystack Enterprise Platform, click Gateway and open the service.
  2. Click Settings.
  3. In the MCP tool section, turn on Use as MCP tool.

The service is immediately available to AI assistants connected to the workspace MCP server. If an AI client sends an optional input, such as filters or top_k, as null or as a copy of its JSON schema, the MCP server ignores it and the service uses the component's default value. For details, see Use Your Pipelines as MCP Tools.

To connect an MCP client to the workspace server, see Use Your Pipelines as MCP Tools.

Test the Service in Gateway​

Send a request straight from the browser to confirm the service responds as you expect.

  1. In Haystack Enterprise Platform, click Gateway and open the service.
  2. Click Connect.
  3. Under Try it, check the API format and response mode. For pipelines with chat or generative output, they match your selections under Endpoint. To test a different combination, change either setting.
  4. Edit the sample request if needed.
  5. Click Send request. A successful run returns the service's response. If the request fails, check that a pipeline version is published to the service, that you have read permission for services, and that your quota isn't used up.

If you edit the sample request and then switch the response mode, the response mode overrides any stream value in the request body. Test requests hit the live service, use the currently serving revision, and count against your quota.

What To Do Next​