Building with Large Language Models (LLMs)

LLMs show remarkable capabilities in understanding and generating human-like text. Have a look at how you can use them in your deepset pipelines.

Suggest Edits

LLMs in Your Pipelines

You can easily integrate LLMs in your deepset pipelines using the versatile Generators coupled with a PromptBuilder. Generators work well for retrieval augmented generation (RAG) question answering and other tasks such as text classification, summarization, and more. It performs the specific task you define within the prompt you pass to it.

Ready-Made Templates for LLM Apps

Pipeline Templates in deepset AI Platform offer a variety of ready-made templates you can use with default settings. Templates with RAG in the title use an LLM and have streaming enabled by default.

In the Conversational category, you'll find RAG Chat templates designed specifically for chat scenarios, such as customer assistants. These chat pipelines include chat history in their responses, ensuring the LLM considers this information to create a human-like conversation experience. All chat pipelines have streaming enabled by default.

Streaming

Streaming refers to the process of generating responses in real time as the model processes input. Instead of waiting for the entire input to be processed before responding, the model generates an answer token by token, making the communication feel more fluid and immediate.

All RAG pipelines in deepset AI Platform have streaming enabled by default.

To read more about streaming and how to enable it, see Streaming.

Tool Calling

Also known as function calling, tool calling is the ability of the model to use external tools to resolve queries. When given the tools, the LLM decides which one to use, and then calls it.

In deepset AI Platform, you can configure tools through ChatGenerators and then call them using ToolInvoker. For details, see Set Up Tool Calling For Your Model.

Creating an LLM App

You can start with an out-of-the-box template and then experiment with models and prompts:

Create a Pipeline from a template. Go to Basic QA templates and choose any RAG template. You can start with RAG Question Answering GPT-4. It uses OpenAIGenerator with the gpt-4-turbo model.
Experiment with your pipeline settings:
1. Try changing the model. You use models through Generator components. Generators are available for various LLMs, including GPT-4, Cohere, Anthropic, or Meta models. For a full list, see Generators.
2. Experiment with different prompts. PromptBuilder is the component that contains the prompt. Learn to write prompts with variables and test them in Prompt Explorer, a sandbox environment in deepset AI Platform, to refine and test your prompts.

Agents

deepset offers an Agent component powered by an LLM of your choice. It can call and run tools, reason, and make autonomous decisions. Try it out in your pipelines or use the pipeline templates we prepared. You can find them on the Pipeline Templates page under Agents. To learn more, see Agents.

Learn More

Here's a collection of information you may want to explore for more information related to LLMs and generative AI in deepset AI Platform.

Building with Large Language Models (LLMs)

LLMs in Your Pipelines

Ready-Made Templates for LLM Apps

Streaming

Tool Calling

Creating an LLM App

Agents

Learn More

About LLMs

Agents

Generative AI in Practice

Prompt Engineering