About This Task

Tool calling, also known as function calling, lets the LLM in your app call a pipeline component. This is useful when the model needs information or capabilities beyond what it knows, such as retrieving real-time data. Here is how it works:

1. You pass the tool configuration to the LLM.
2. The LLM generates a JSON schema with all parameters and values needed to call the tool, but the model does not call the tool itself.
3. The LLM sends the tool call to ToolInvoker that runs the tool.
4. You then send the messages from ToolInvoker back to the LLM so that it can use them to resolve the query.

Tool calling is supported by chat models used through ChatGenerators. For detailed information on chat models and tool calling, see Chat models. For information on using chat models in deepset AI Platform, see Generators Overview.

Prerequisites

• Ensure the model you're using supports tool calling.
• To learn about the Haystack Tool data class, see Haystack documentation.

Set Up the Tool Call

ℹ️

If you're using a custom component, first check if it works correctly in deepset AI Platform.

1. Add ToolInvoker to your pipeline.
2. On the ToolInvoker card, click Configure and add the following information to the YAML:
   ToolInvoker configuration for a component

   - type: haystack.tools.component_tool.ComponentTool # this is a component wrapper that allows components to be used as tools
     data:
       component:
         type: # the import path to the component, to quickly check a component type, add it to your pipeline and switch to the YAML view
         init_parameters: # configure your component, pass all init parameters here
           parameter1: value
           paramter2: value
       name: # give your component a name, you can refer to it in the prompt
       description: # describe what the component does, it will help the LLM decide when to use it
   

This is an example of how to configure SerperDevWebSearch as a tool for the LLM:

- type: haystack.tools.component_tool.ComponentTool
  data:
    component:
      type: haystack.components.websearch.serper_dev.SerperDevWebSearch
      init_parameters:
        api_key:
          type: env_var
          env_vars:
            - SERPERDEV_API_KEY
          strict: false
        top_k: 10
    name: web_search
    description: Search the web for current information on any topic

3. Configure a ToolInvoker for each tool you want to add.

4. Configure the ChatGenerator:

   1. Expand optional parameters on the ChatGenerator card and click Configure under tools:

      

   2. Add the same configuration as you added in ToolInvoker. List each tool you want to give the LLM:

      ChatGenerator tools configuration

      # tool 1
      - type: haystack.tools.component_tool.ComponentTool # this is a component wrapper that allows components to be used as tools
        data:
          component:
            type: # the import path to the component, to quickly check a component type, add it to your pipeline and switch to the YAML view
            init_parameters: # configure your component, pass all init parameters here
              parameter1: value
              paramter2: value
          name: # give your component a name, you can refer to it in the prompt
          description: # describe what the component does, it will help the LLM decide when to use it
      # tool 2
      - type: haystack.tools.component_tool.ComponentTool a
        data:
          component:
            type: # component type
            init_parameters: # component configuration
              parameter1: value
              paramter2: value
          name: # give your component a name, you can refer to it in the prompt
          description: # describe what the component does, it will help the LLM decide when 
      

5. Ensure you mention the tools in ChatPromptBuilder's prompt, for example:

   ChatPromptBuilder prompt with tool information

   - _content:
       - text: |
           You are a helpful assistant answering the user's questions based on the provided documents.
           If the answer is not in the documents, rely on the web_search tool to find information.
           Do not use your own knowledge.
     _role: system
   

Depending on the model, you may find that some rely more on the instructions in the prompt when choosing a tool, while others on the tool descriptions.

6. Ensure each ToolInvoker receives the ChatGenerator's replies. You can either connect the ChatGenerator's replies output directly to ToolInvoker's messages or if you're using any component in between, ensure it passes the replies on to ToolInvoker.
7. Pass each TooInvoker's messages back to theChatGenerator so that the LLM can use them to resolve the query.
   Important: The LLM must also receive the tool call it sent to the tool. The tool call must precede the message from the tool. You can use OutputAdapter to convert all the messages into a list of ChatMessage objects.