Check the init and runtime parameters you can use to customize CohereGenerator.
YAML Init Parameters
These are the parameters you can pass to this component in the pipeline YAML configuration:
| Parameter | Type | Possible values | Description |
|---|---|---|---|
api_key | Secret | Default: {"type": "env_var", "env_vars": ["COHERE_API_KEY", "CO_API_KEY"], "strict": False} | The API key for the Cohere API. Required. |
model | String | commandcommand-lightcommand-nightlycommand-nightly-lightDefault: command | The name of the model to use. Required. |
streaming_callback | Callable | Default: None | A callback function to be called with the streaming response. Optional. |
api_base_url | String | Default: None | The base URL of the Cohere API. Optional. |
kwargs | Dictionary of string and any | Additional model parameters used during generation. Refer to Cohere API documentation for more details. Some supported parameters include: - max_tokens: The maximum number of tokens to be generated. Defaults to 1024.- truncate: Specifies how the API handles inputs longer than the maximum token lenght. Possible values are NONE, START, END. Defaults to END.- temperature: A float that tunes the degree of randomness in generation. Lower temperature means less random generation.- preset: Identifier of a custom preset. A preset is a combination of parameters, such as prompt, temperature, and so on. You can create presets in the playground.- end_sequences: The generated text is cut at the beginning of the earliest occurrence of an end sequence. The sequence is excluded from the text.- stop_sequences: The generated text is cut at the end of the earliest occurrence of a stop sequence. The sequence is excluded from the text.- k: Defaults to 0. The minimum value is 0.01 and the maximum values is 0.99.- p: Ensures that only the most likely tokens with the total probability mass of p are considered for generation at each step. If both k and p are enabled, p acts after k.- frequency_penalty: Reduces the repetitiveness of generated tokens. The higher the value, the stronger a penalty is applied to the previously present tokens, proportional to the number of times they have already appeared in the prompt or prior generation.- presence_penalty: Defaults to 0.0, with the minimum value of 0.0 and a maximum value of 1.0. You can use it to reduce the repetitiveness of the generated tokens. This penalty is applied equally to all tokens that have already appeared, regardless of their frequency.- return_likelihoods: Specifies if token likelihoods are returned in the response. Possible values are NONE, GENERATION, and ALL. Defaults to NONE.- logit_bias: Prevents the model from generating unwanted tokens or ot incentivize it to include desired tokens. The format is {token_id: bias} wherebias is a float between -10 and 10.Optional |
REST API Runtime Parameters
There are no runtime parameters you can pass to this component when making a request to the Search REST API endpoint.