About This Task

Secrets let you connect to providers not listed on the Integrations page. You can also use them for custom components or to manage production and development API keys.

For more information about secrets, see Secrets and Integrations.

Secret Scope

You can add secrets at either the workspace or organization level:

• Workspace-level secrets are available only to pipelines and indexes within that specific workspace.
• Organization-level secrets are available to all workspaces in the organization.

If the same provider is integrated at both levels, the workspace-level integration takes priority.

Prerequisites

• Make sure you have an active account with the provider.
• Have your API key or access credentials ready.
• For existing components: Check the component's documentation to find the environment variable it uses to read the API key.
• For custom components: In your component's code, define and environment variable to store the API key. Then, create a secret with he same name.

Connect to a Provider

Add Workspace-Level Secrets

1. Click your profile icon and choose Settings.
2. Go to Workspace>Secrets.
3. Click Create Secret.
4. Enter a secret name that matches the environment variable to store the API key.
5. Enter the API key and any other required details.
6. Click Create. You can use this connection in pipelines and indexes in the current workspace.

Add Organization-Level Secrets

1. Click your profile icon and choose Settings.
2. Go to Organization>Secrets.
3. Enter a secret name that matches the environment variable to store the API key.
4. Enter the API key and any other required details.
5. Click Create. You can use this connection in pipelines and indexes in all workspaces in the current organization.

Use Secrets in Your Custom Components

Add the secret to your component's code:

• Make sure you import it.
• Specify the environment variable name, which must match the secret's name.
• Add its serialization and deserialization methods.
• Add a warm_up() method to load the API key before pipeline validation. This step is optional but we recommend it:

from haystack import component, default_from_dict, default_to_dict
from haystack.utils import Secret, deserialize_secrets_inplace
from typing import Any, Dict


@component
class MyComponent:
    def __init__(self, model_name: str, api_key: Secret = Secret.from_env_var("ENV_VAR_NAME")):
        self.model_name = model_name
        self.api_key = api_key
        self.backend = None

    def warm_up(self):
        # Call api_key.resolve_value() to load the API key from the environment variable
        # We put the resolution in warm_up() to avoid loading the API key during pipeline validation
        if self.backend is None:
            self.backend = SomeBackend(self.model_name, self.api_key.resolve_value())

    def to_dict(self) -> Dict[str, Any]:
        # Make sure to include any other init parameters in the to_dict method
        return default_to_dict(
            self,
            model_name=self.model_name,
            api_key=self.api_key.to_dict(),
        )

    @classmethod
    def from_dict(cls, data: Dict[str, Any]) -> "MyComponent":
        # Make sure to use deserialize_secrets_inplace to load the Secret object
        init_params = data.get("init_parameters", {})
        deserialize_secrets_inplace(init_params, keys=["api_key"])
        return default_from_dict(cls, data)

    def run(self, my_input: Any):
        if self.backend is None:
            raise RuntimeError("The component wasn't warmed up. Run 'warm_up()' before calling 'run()'.")
        return self.backend.process(my_input)

If you have multiple secrets for one provider, you can easily switch between them in your pipeline YAML by updating the secret's name:

llm:
  type: dc_custom_component.components.my_components.component1.MyComponent # the path to your component
  init_parameters:
     api_key: {"type": "env_var", "env_vars": ["ENV_VAR_NAME"], "strict": False} # uses the `ENV_VAR_NAME` secret
     # to use another secret, update its name here