YAML Init Parameters

These are the parameters you can pass to this component in the pipeline YAML configuration:

Parameter	Type	Possible values	Description
`model`	String		The name of the Sagemaker model endpoint. Required.
`aws_access_key_id`	Secret	Default: `{"type": "env_var", "env_vars": ["AWS_ACCESS_KEY_ID"], "strict": False}`	The `Secret` for AWS access key ID. Optional.
`aws_secret_access_key`	Secret	Default: `{"type": "env_var", "env_vars": ["AWS_SECRET_ACCESS_KEY"], "strict": False}`	The `Secret` for AWS secret access key. Optional.
`aws_session_token`	Secret	Default: `{"type": "env_var", "env_vars": ["AWS_SESSION_TOKEN"], "strict": False}`	The `Secret` for AWS session token. Optional.
`aws_region_name`	Secret	Default: `{"type": "env_var", "env_vars": ["AWS_DEFAULT_REGION"], "strict": False}`	The `Secret` for AWS region name. If not provided, the default region is used. Optional.
`aws_profile_name`	Secret	Default: `{"type": "env_var", "env_vars": ["AWS_PROFILE"], "strict": False}`	The `Secret` for AWS profile name. If not provided, the default profile is used. Optional.
`aws_custom_attributes`	Dictionary of string and any	Default: `None`	Custom attributes to be passed to SageMaker, for example `{"accept_eula": True}` for Llama2 models. Optional.
`generation_kwargs`	Dictionary of string and any	Default: `None`	Additional keyword arguments for text generation. For a list of supported parameters, see your model's documentation page. For example, for Hugging Face models, see: Run inference. Llama2 models support the following inference payload parameters: - `max_new_tokens`: The model generates text until the output length (excluding the input context length) reaches `max_new_tokens`. If specified, it must be a positive integer. - `temperature`: A float that controls the randomness in the output. Higher temperature results in output sequence with low-probability words and lower temperature results in output sequence with high-probability words. If `temperature=0`, it results in greedy decoding. If specified, it must be a positive float. - `top_p`: In each step of text generation, sample from the smallest possible set of words with cumulative probability `top_p`. If specified, it must be a float between 0 and 1. - `return_full_text`: If `True`, input text is a part of the output generated text. Optional.

REST API Runtime Parameters

There are no runtime parameters you can pass to this component when making a request to the Search REST API endpoint.