api_key

Secret

Default: {"type": "env_var", "env_vars": ["COHERE_API_KEY", "CO_API_KEY"], "strict": False}

The API key for the Cohere API.
Required.

model

String

command
command-light
command-nightly
command-nightly-light
Default: command

The name of the model to use. Required.

streaming_callback

Callable

Default: None

A callback function to be called with the streaming response. Optional.

api_base_url

String

Default: None

The base URL of the Cohere API. Optional.

kwargs

Dictionary of string and any

Additional model parameters used during generation. Refer to Cohere API documentation for more details. Some supported parameters include:

• max_tokens: The maximum number of tokens to be generated. Defaults to 1024.
• truncate: Specifies how the API handles inputs longer than the maximum token lenght. Possible values are NONE, START, END. Defaults to END.
• temperature: A float that tunes the degree of randomness in generation. Lower temperature means less random generation.
• preset: Identifier of a custom preset. A preset is a combination of parameters, such as prompt, temperature, and so on. You can create presets in the playground.
• end_sequences: The generated text is cut at the beginning of the earliest occurrence of an end sequence. The sequence is excluded from the text.
• stop_sequences: The generated text is cut at the end of the earliest occurrence of a stop sequence. The sequence is excluded from the text.
• k: Defaults to 0. The minimum value is 0.01 and the maximum values is 0.99.
• p: Ensures that only the most likely tokens with the total probability mass of p are considered for generation at each step. If both k and p are enabled, p acts after k.
• frequency_penalty: Reduces the repetitiveness of generated tokens. The higher the value, the stronger a penalty is applied to the previously present tokens, proportional to the number of times they have already appeared in the prompt or prior generation.
• presence_penalty: Defaults to 0.0, with the minimum value of 0.0 and a maximum value of 1.0. You can use it to reduce the repetitiveness of the generated tokens. This penalty is applied equally to all tokens that have already appeared, regardless of their frequency.
• return_likelihoods: Specifies if token likelihoods are returned in the response. Possible values are NONE, GENERATION, and ALL. Defaults to NONE.
• logit_bias: Prevents the model from generating unwanted tokens or ot incentivize it to include desired tokens. The format is {token_id: bias} wherebias is a float between -10 and 10.
  Optional