Add Custom Model Definitions
Configure custom model configurations to use in your pipelines. Custom models let you define YAML configurations for models and reuse them across pipelines.
About This Task
A custom model definition is a reusable configuration for a chat model, for example a model with your preferred parameters or one served from a custom endpoint. You can create it for a workspace or for your whole organization. For what custom models are, which model types they support, and how they override other models, see Custom Models.
Prerequisites
- You must be an organization Admin to add custom model definitions for an organization, or have write access to a workspace to add custom model definitions to a workspace.
- You must have a valid API key from the model provider.
Add a Custom Model Definition
- In Haystack Enterprise Platform, click your profile icon and choose Settings.
- Go to Organization>Integrations (for organization-level custom models) or Workspace>Integrations (for workspace-level custom models).
- Find Custom Models and click Configure.
- Click Add model.
- Enter the following details:
- Model name: A descriptive name for your custom model
- Provider: Pick the model provider from the list. For a provider that is not listed, choose Custom.
- Configure the YAML settings:
- Chat Generator: Define the component configuration including API endpoints and authentication.
- Model Parameters: Specify model parameters, like temperature and max output tokens. These parameters are specific to the model provider you choose. Check the model provider's documentation for allowed parameters.
- Optionally, try the model to check if the configuration is working.
- Save your changes.
Add a DeepSeek Model
Before you start, add a secret named DEEPSEEK_API_KEY that holds your DeepSeek API key. Then, when you add the custom model, choose DeepSeek as the Provider. DeepSeek serves an OpenAI-compatible API, so the Chat Generator configuration is pre-filled with the DeepSeek endpoint (https://api.deepseek.com), the model deepseek-flash, and an API key read from a secret named DEEPSEEK_API_KEY. To use another DeepSeek model, change the model value. Check DeepSeek's documentation for current model IDs.
The platform doesn't check that the secret exists when you save the model, so a DeepSeek model can show as connected before you add the secret. Add the secret before you use the model in a pipeline. To check the setup, try the model before you save it. If it fails, make sure the DEEPSEEK_API_KEY secret exists and holds a valid key.
Use Custom Models in Pipelines
After you save a custom model definition, choose it from the list of available models when you configure an AI component, like LLM or Agent. The model is shown alongside standard models in component configuration.
When you add a custom model to a pipeline, the pipeline gets a copy. To update the model in the pipeline, edit the pipeline YAML configuration. For details, see Custom Models.

Manage Custom Models
You can update or delete custom model definitions. Changes you make to a definition don't affect models already copied into your pipelines.
- On the Integrations page, click Configure on the Custom Models Card. All custom model definitions are listed here.
- Click Edit next to the model definition you want to update, or Delete to remove it.
Duplicate and Overlapping Models
If saving fails because the model duplicates another one, or a custom model replaced a platform model, see Model Overrides.
Retry When Another Change Is in Progress
The platform saves model changes in an organization one at a time, so two people can't create the same model at once. If another model change in your organization is still running, your save can fail with HTTP status 503 and a message that another change is in progress. Wait about 5 seconds and save again. If you call the API directly, the response includes a Retry-After header with the number of seconds to wait. For the endpoints, see Create Organization Model, Update Organization Model, and Create Workspace Model.
If the error keeps appearing after several retries, a model change in your organization may be stuck. Contact support.
Related Information
Was this page helpful?