Building with Large Language Models (LLMs)
LLMs show remarkable capabilities in understanding and generating human-like text. Have a look at how you can use them in your deepset pipelines.
LLMs in Your Pipelines
You can easily integrate LLMs in your deepset pipelines using the versatile Generators coupled with a PromptBuilder or a ChatPromptBuilder. Generators work well for retrieval augmented generation (RAG) question answering and other tasks such as text classification, summarization, and more. It performs the specific task you define within the prompt you pass to it.
To build autonomous systems that make decisions, use the Agent component. It uses a ChatGenerator as its decision-making brain and can call tools you provide.
For a deeper explanation of decision-making systems, check Agentic Pipelines.
Agents
deepset offers an Agent component powered by an LLM of your choice. It can call and run tools, reason, and make autonomous decisions. Try it out in your pipelines or use the pipeline templates we prepared. You can find them on the Pipeline Templates page under Agents. To learn more, see Agents.
Ready-Made Templates for LLM Apps
Pipeline Templates in deepset AI Platform offer a variety of ready-made templates you can use with default settings. Templates with RAG in the title use an LLM and have streaming enabled by default.
In the Conversational category, you'll find RAG Chat templates designed specifically for chat scenarios, such as customer assistants. These chat pipelines include chat history in their responses, ensuring the LLM considers this information to create a human-like conversation experience. All chat pipelines have streaming enabled by default.
There's also a group of Agent templates with tools such as web search and RAG already configured. Check the Agent group.
Streaming
Streaming refers to the process of generating responses in real time as the model processes input. Instead of waiting for the entire input to be processed before responding, the model generates an answer token by token, making the communication feel more fluid and immediate.
All RAG pipelines in deepset AI Platform have streaming enabled by default.
To read more about streaming and how to enable it, see Streaming.
Tool Calling
Also known as function calling, tool calling is the ability of the model to use external tools to resolve queries. When given the tools, the LLM decides which one to use, and then calls it.
In deepset AI Platform, you can configure tools through ChatGenerators and then call them using ToolInvoker, or you can add an Agent component and configure its tools. For details, see Set Up Tool Calling For Your Model and Agent.
Learn More
Here's a collection of information you may want to explore for more information related to LLMs and generative AI in deepset AI Platform.
About LLMs
Agents
Generative AI in Practice
- Generators
- Tutorial: Building a Robust RAG System
- Enable References for Generated Answers
- Enable Streaming
- Using Hosted Models and External Services
Prompt Engineering
Updated about 16 hours ago