Components of an AI-Based App

Here are the typical components of an LLM-based app:

Knowledge Base

A repository of information your app can reference to provide accurate and relevant responses. This can be a simple document store with text data, a vector store for embeddings, or a structured database.

Context Retrieval

This component finds relevant information to give context to the model. If your app uses existing data rather than user-uploaded content, you’ll need context retrieval, like a search function. This may be a simple keyword search, semantic search, or both.

You might also include a ranking tool to prioritize information. For example, you could highlight the latest data or entries with specific metadata.

The type of context retrieval you choose depends on your app's purpose and desired output.

Prompt Rendering

This component builds effective prompts by combining user input, retrieved context (like documents), and predefined templates with placeholders for dynamic content. The rendered prompt then goes to the LLM for processing.

LLM Integration

This is the interface with the large language model, either through an API or a local deployment. It takes the rendered prompt and answers queries based on the instructions. It’s the heart of your application.

Consider trying different models to balance performance and costs. We recommend making this component flexible enough to switch between models easily.

Decision Component

Depending on your system requirements, you might want a component that chooses different paths based on the data it receives. For example, in a customer service app, you could route happy customers to a chatbot and upset customers to a human agent.

Output Processing

This component formats and potentially filters LLM’s responses. It might involve text cleaning or creating objects that your interface can easily display.

Feedback Collection

This crucial component gathers and stores user feedback. It helps you understand how well your app performs and what needs improvement. Make sure your test users can easily share their thoughts on your system's answers.