Basic Concepts

Component

Components are building blocks of pipelines. Each component performs a specific task in a pipeline. For example, PromptBuilder is a component that inserts variables and documents into a prompt and sends it to the Generator component that uses an LLM to perform the instructions from the prompt.
Components in a pipeline act like building blocks that you can mix and match or replace. For a list of available components, see Pipeline Components.

Document

Refers to an individual piece of text stored in the document store. Multiple documents may originally come from one file. Documents are created during indexing, when the files are preprocessed, cleaned, split, and converted into passages of text. Pipelines use documents at search time. They don't work on files stored in deepset Cloud, but retrieve documents from the document store and run searches and other tasks on them.

Document Store

A database that stores the text documents, their metadata, and (optionally) embeddings. At search time, Retrievers in your pipeline search the document store to find documents that are the most relevant to the query. To check the databases deepset Cloud supports, see Document Stores.

File

Refers to the raw file you upload to deepset Cloud (for example, a PDF). When an indexing pipeline runs, files get converted, cleaned, and split into documents, which contain the actual text and are then used by pipelines for finding the best answer to a query.

Groundedness Observability

A dashboard where you can check the groundedness score of your RAG pipelines. This score tells you how grounded in your documents the LLM's answers are.

Indexing

It refers to a process of preprocessing your files, turning them into documents, and then storing those documents in the document store. Indexing happens after you deploy a pipeline. The exact indexing steps are defined in the indexing pipeline (for example, the size of the documents resulting from a file).

Organization

In deepset Cloud, your company is assigned an organization. Within each organization, there are up to 10 workspaces. When you invite users to deepset Cloud, they gain access to your organization and all the workspaces within it.

Pipeline

In deepset Cloud you work with pipelines. A pipeline is an app that defines the flow of data. There are two types of pipelines - query and indexing. Query pipelines define the steps that happen after the user asks a query until they get an answer. Indexing pipelines define the steps to clean, split, and convert your files into documents the query pipeline can use to find answers.

Pipelines are made up of components that define the processing steps. Components in pipelines are connected by their input and output, so that the output of one component is compatible with the input of another one. You can mix and match the components in a pipeline.

Workspace

In deepset Cloud, you work in workspaces, where you upload your data and create and maintain your pipelines. Data and pipelines are not shared across workspaces.

All workspaces belong to an organization. When you invite people to your organization, they automatically receive access to all workspaces within this organization. You cannot limit access to a workspace.