Custom Models
A custom model is a reusable configuration for a model that you define once and use across your pipelines.
What Are Custom Models?
A custom model definition is a reusable configuration for a chat model. It can include a specific model, its parameters, or both. For example, you can define an OpenAI model with custom parameters and reuse it across multiple pipelines. With custom models, you can:
- Use models from providers beyond the standard integrations.
- Configure model parameters and generation settings for your use case.
- Create reusable model configurations for your organization or workspace.
- Set custom API endpoints and authentication methods.
You create custom models at two levels:
- Workspace: Available only in the current workspace.
- Organization: Available across all workspaces in your organization.
How You Use Custom Models
You add a custom model definition on the Integrations page in Settings. After you save it, the model appears in the list of available models on components that support models, such as LLM or Agent, next to the standard models. You pick it from the list like any other model. For the steps, see Add Custom Model Definitions.
When you add a custom model to a pipeline, the pipeline gets a copy of the model. Changes you later make to the definition on the Integrations page don't affect the model in your pipeline. To update the model in the pipeline, edit the pipeline YAML configuration.
When you switch an LLM or Agent from a platform OpenAI model to a custom model that sets its own API endpoint, such as DeepSeek or another OpenAI-compatible model, the component uses the custom model's own generator class and configuration instead of keeping the OpenAI class. Requests then go to the custom endpoint without extra setup, which matters because endpoints such as DeepSeek's don't serve the OpenAI Responses API.
Supported Model Types
Custom models are chat models only: models that generate text, used by LLM and Agent. A custom model definition is always a chat generator configuration. You can point it at the providers and endpoints that the chat generator configuration supports.
You can't add an embedding model or a ranker as a custom model. For example, a reranker served with vLLM can't be added this way. To use one, configure it directly on the component in your pipeline, such as VLLMRanker or VLLMTextEmbedder, with the server URL and a secret for the API key. For the parameters, see VLLMRanker and VLLMTextEmbedder. The embedders and rankers in the model list are the ones deepset hosts.
Model Overrides
A custom model can override another model with the same provider component and model name. The sections below explain how the platform checks for duplicates and which model it shows in the model list.
Duplicate Models
A workspace model can have an organization-level counterpart: a model with the same provider component and model name. As with secrets and integrations, the workspace model overrides its organization counterpart, so you can keep editing the workspace model.
The platform checks for duplicates when you save a model:
- Adding an organization model: The platform compares it only with other organization models. A workspace model with the same component and model name doesn't block it.
- Adding a workspace model: Saving fails if the same component and model name already exist in that workspace or at the organization level. To resolve it, change the model name or edit the existing model instead.
- Editing a model: The platform checks for duplicates only if you change the provider component or the model name. Changes to other settings, such as the display name or model parameters, are always allowed, even if the model has a counterpart at another level. If you change the component or model name so that a workspace model matches another model in its workspace or at the organization level, saving fails. An organization model is compared only with other organization models.
Find Models That Hide Other Models
If a custom model doesn't behave as expected, check whether it overrides another model with the same provider component and model name. To find out, go to Settings > Organization>Integrations or Workspace>Integrations, click Configure on Custom Models, and check the Name column. A tag under the model name shows what the model hides:
- Hides organization model: Shown in the workspace list. An organization custom model has the same provider component and model name, so this workspace model takes precedence over it in the workspace.
- Hides platform model: Shown in the workspace and organization lists. A platform model has the same provider component and model name, so this custom model takes precedence over it.
A model can show both tags in the workspace list.
Two models match only if they use the same provider component and the same model name or, when a model has no model name, the same Azure deployment name. The display name doesn't matter, and a model without a model name never matches.
The organization list doesn't show which workspace models override an organization model.
How Overlapping Models Are Listed
If a platform model is missing from the model list, or your custom model replaced it, a model with the same identity is overriding it. The model list in a workspace shows each model once. When models share the same provider component and model name (or Azure deployment name), the list keeps the workspace model over the organization model, and the organization model over the platform model. If two models in the same level match, the newest one is kept.
When you reopen an Agent or LLM component, the picker shows the model the component actually uses. For example, if your workspace has a custom gpt-4o that overrides the platform gpt-4o, you see only the custom one.
- Model variants stay separate: The same model used through a different generator class of the same provider, for example
OpenAIChatGeneratorandOpenAIResponsesChatGenerator, is listed as separate entries. Each is a different configuration that a component can select. - Connection status doesn't change the result: A workspace model that isn't connected still hides the platform model it overrides. If you filter the list to connected models, the hidden platform model doesn't reappear.
- Model parameters can be hidden: When you copy generation parameters from an existing model while defining a custom model, the list offers only the models that remain after deduplication. A custom model with no generation parameters that overrides a platform model hides that platform model's parameters. To copy them, remove the override, change its component or model name in the Chat Generator configuration, or enter the parameters manually.
If you call the API directly, Get Workspace Models returns the deduplicated list, and total counts the models after deduplication.
Related Information
Was this page helpful?