Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Use Your Services as MCP Tools

Turn on your published services as tools on a Gateway MCP server, and connect AI assistants like Claude Code or Cursor to call them.


About This Task​

MCP (Model Context Protocol) standardizes how AI applications communicate with external tools. In Haystack Enterprise Platform, a Gateway MCP server exposes the services you choose as tools. Currently, each workspace has one Gateway MCP server. A new Gateway MCP server has no tools. You turn on each service individually.

A service is the stable endpoint under which a pipeline runs. When an AI assistant calls a tool, the service runs the pipeline behind it. A pipeline must be published as a service before it can be a tool. When you publish a new pipeline version to the service, the tool uses it and the endpoint stays the same, so you don't need to update your AI client. For details, see Gateway.

For example, you can add a RAG service as a tool so your AI assistant can query your internal knowledge base. To learn more about MCP, see Model Context Protocol.

Prerequisites​

  • One or more published services in your workspace. For instructions, see Publish a Pipeline to a Service.
  • The MCP Server permission for the workspace with write access, and the Gateway permission with at least read access, so that you can open services. Admin and Editor roles have read and write access by default. Search User role has read-only access by default. Users on custom roles won't see MCP features until an admin grants the MCP Server permission to that role. For details, see User Roles and Permissions.

Create a Gateway MCP Server​

Currently, you create one Gateway MCP server per workspace. Creating the server doesn't expose any services yet.

  1. In Haystack Enterprise Platform, click your profile icon and go to Settings > Workspace > MCP.
  2. Click Create MCP Server.
  3. Type a name for the MCP server.
  4. Optionally, provide the following information:
    • The server version. Your AI assistant sends the version number in the initial call to the server. Leave this field blank to use the latest version.
    • Website URL. The URL of the company that built the MCP server.
    • Icon URL. The URL of a custom icon for the MCP server, in SVG or PNG format. Not all AI assistants can display custom icons.
    • Instructions. Guidance for the AI assistant on how to use your tools. The text is added to the assistant's prompt at the start of every session, so it applies to all tools. Give instructions beyond the tool descriptions, such as the order for calling services, dependencies between them, or known limitations.

Turn On a Service as a Tool​

  1. In Haystack Enterprise Platform, click Gateway and open the service.
  2. Click Settings.
  3. In the MCP tool section, turn on Use as MCP tool. If the workspace has no MCP server yet, it's created for you.
  4. Optionally, enter a tool name. Leave the field empty to use the service name. Names help the AI assistant choose the right tool, so make sure the name is unique and descriptive.
  5. Optionally, enter a tool description. Leave the field empty to have a description generated automatically. Descriptions tell the AI assistant when and how to use the tool.

The service is immediately available to AI assistants connected to your Gateway MCP server. Repeat these steps for each service you want to expose.

Updating the Tool Name

If you change the tool name in a service's MCP tool settings, the tool name that AI clients see changes as well. Clients that refer to the old name need to be updated.

Connect Your AI Client​

All clients connect to the URL of your Gateway MCP server. Currently, the URL contains your workspace ID:

https://api.cloud.deepset.ai/api/v2/workspaces/<your-workspace-id>/mcp

Replace <your-workspace-id> with the ID of your workspace. To find it, see Find Your Workspace ID. If you use the US cloud, replace api.cloud.deepset.ai with api.us.deepset.ai.

You can sign in with your Haystack Platform user account (OAuth) or use an API key.

OAuthAPI key
Sign-inIn the browser, with your user accountNone. The key goes in the request header.
OrganizationsAll organizations your account belongs to. You can add an organization ID to address a specific one.The organization the key belongs to

Connect with OAuth​

In the examples, replace the placeholders with your IDs:

  • <your-workspace-id>: The ID of your workspace. To find it, see Find Your Workspace ID.
  • <your-organization-id>: The ID of your organization. You only need it if you belong to several organizations. To find it, see Find Your Organization ID.

For Claude Code:

claude mcp add --transport http haystack-gateway \
https://api.cloud.deepset.ai/api/v2/workspaces/<your-workspace-id>/mcp \
--header "X-Organization-ID: <your-organization-id>"

Then run /mcp in Claude Code, select haystack-gateway, choose Authenticate, and sign in with your Haystack Platform account in the browser.

For Cursor, add the URL as a remote server:

{
"mcpServers": {
"haystack-gateway": {
"url": "https://api.cloud.deepset.ai/api/v2/workspaces/<your-workspace-id>/mcp",
"headers": {
"X-Organization-ID": "<your-organization-id>"
}
}
}
}

The X-Organization-ID header is optional. Add it if your account belongs to several organizations, so that the server addresses the organization that owns the workspace. If you belong to only one organization, leave it out.

Connect with an API Key​

In the examples, replace <your-workspace-id> with the ID of your workspace. To find it, see Find Your Workspace ID. For instructions on creating <your-api-key>, see Generate API Keys.

For Claude Code:

claude mcp add --transport http haystack-gateway \
https://api.cloud.deepset.ai/api/v2/workspaces/<your-workspace-id>/mcp \
--header "Authorization: Bearer <your-api-key>"

For Cursor:

{
"mcpServers": {
"haystack-gateway": {
"url": "https://api.cloud.deepset.ai/api/v2/workspaces/<your-workspace-id>/mcp",
"headers": {
"Authorization": "Bearer <your-api-key>"
}
}
}
}

An API key belongs to one organization, so you don't need the X-Organization-ID header with an API key.

When the AI client connects, it receives the list of services available as tools and can call them as needed.

Tool Inputs​

The inputs of a tool come from the inputs of the pipeline that runs behind the service. The tool publishes the inputs that you declare in the inputs section of the pipeline YAML. It also publishes any required input that isn't connected to another component. A pipeline that takes a query publishes a query string, and a pipeline that takes messages publishes messages. The tool doesn't add inputs your pipeline doesn't have.

Pipelines built around an Agent or an LLM, including pipelines that use Anthropic models, take a list of chat messages as input. These pipelines publish a required messages input. It's an array of objects, and each object has these fields:

  • role: Who sent the message. Use user or assistant.
  • content: The text of the message, as a string.

For example, an AI assistant can send this value for messages:

[
{"role": "user", "content": "What is our parental leave policy?"}
]

To hold a multi-turn conversation, include earlier messages in the array, alternating between user and assistant roles. The pipeline converts each object into a chat message before it runs.

Inputs with complex types that have no simple tool format aren't published. If an input like this is required, the tool can't be called with it. Use inputs of simple types, such as strings, numbers, booleans, lists, or chat messages.

Optional inputs, such as filters or top_k, don't need a value in every call. Some AI clients send an optional input as null, or copy the input's JSON schema (for example, {"anyOf": [{"type": "object"}, {"type": "null"}]}) into the call instead of a real value. The MCP server drops these arguments, so the component uses its own default value instead of receiving a placeholder it can't use. You don't need to change your pipeline. To apply a specific value, make sure your AI assistant sends an actual value for the input. An object that only has a type key, such as {"type": "pdf"}, is treated as a real value and is passed on.

Turn Off a Service Tool or Delete the Gateway MCP Server​

Turn Off a Service Tool​

  1. In Haystack Enterprise Platform, click Gateway and open the service.
  2. Click Settings.
  3. In the MCP tool section, turn off Use as MCP tool.

The service is immediately removed from the server. Other tools continue to work.

Delete the Gateway MCP Server​

  1. Go to Settings > Workspace > MCP.
  2. Switch off Create MCP Server.

The server and all its tools are immediately removed and can no longer be used by external clients.

Troubleshooting​

  • The MCP section is grayed out: You need write access to the MCP Server permission. Contact your workspace administrator.

  • The tool toggle is disabled: Hover over the toggle to see the reason. Common causes are that the service isn't published or active, or that you don't have write access to the MCP server.

  • AI client can't connect: Check the workspace ID in the URL. With an API key, check that the key is valid and has access to the workspace. With OAuth, check that your account has access to the workspace and, if you belong to several organizations, that you send the X-Organization-ID header with the ID of the organization that owns the workspace. This makes sure you address the correct organization.

  • Changes aren't saving: Settings are saved automatically with a short delay. Wait a few seconds before navigating away.

  • The tool call fails with "requires a non-empty 'query'": The AI assistant called the tool with a blank query and no other input, so there was nothing for the pipeline to run on. This is expected: provide a non-empty query, or make sure at least one other input (for example, filters or a file ID) is included in the call. A service that relies only on its other inputs keeps working with an empty query.

  • The tool for an Agent or LLM pipeline doesn't list a messages input: Make sure the pipeline declares messages in its inputs section and connects it to the Agent or LLM messages input. Then publish the new version to the service.

  • A required input is missing from the tool: The input has a complex type that has no simple tool format, so it isn't published. Change the pipeline so that the input uses a simple type, such as a string, or connect it to another component.

  • An optional input such as filters or top_k seems to be ignored, or the tool call used to fail with an 'operator' key missing error or an OpenSearch 400 error: The AI client sent null or a copy of the input's JSON schema instead of a value. The MCP server drops these arguments, so the pipeline runs with the component's default value. To apply a value, make sure the AI assistant sends one, for example, a filter object with field, operator, and value keys.

  • Answers over MCP feel slow or don't stream: Your Gateway MCP server returns each answer in one piece after the pipeline finishes, and the first call after the idle timeout waits for a cold start. For details and tips, see Response Time and Streaming.

  • The tool call fails with "The pipeline instance stopped before it finished responding": The pipeline instance closed the connection before it returned a result. Wait a minute, then call the tool again. If the error keeps happening, contact deepset support.