About this Task

Currently, the deepset AI Platform supports the following document stores:

• OpenSearch (the core document store, unless you're on your own OpenSearch cluster, you don't need to provide any credentials and can just use it out of the box)
• Elasticsearch
• MongoDB
• Pinecone
• Qdrant
• Snowflake (for details on connecting to Snowflake, see Connect to Your Snowflake Database)
• Weaviate

These databases act as document stores in the deepset AI Platform. For more information, see Document Stores.

You can also add an integration with any other database through a custom component. For details, see Custom Components.

Prerequisites

• You need an active API key to the database you want to use.
• Basic knowledge of document stores in deepset. For details, see Document Stores.
• Understanding of secrets in deepset. To learn more, see Add Secrets to Connect to Third Party Providers.
• Check the parameters you can configure for your document store. See the documentation for your document store in the Document Stores section.

Run Queries on Data in Your Document Store

This task involves three steps:

1. Create a secret to securely store your API key to the document store.
2. Write your documents to the document store.
3. Retrieve documents from the document store.

Create a Secret for Your Document Store

📘

MongoDB

To use MongoDBAtlasDocumentStore, you can simply paste your API key on the Connections page:

1. Click your initials in the top right corner and choose Connections.
2. Find MongoDB, click Connect, and paste the API key.

This step is needed to enable a connection with the database you want to use as the document store without adding the API key explicitly in the configuration.

1. In the deepset AI Platform, click your initials in the top right corner and choose Secrets>Add New Secret.

   

2. Give your secret the same name as the environment variable where you want to store it.

3. Paste the API key into the Secret field and save it.

You'll then use the secret name as the API key for components that need to connect to the document store.

Write Documents into the Document Store

DocumentWriter is the component that writes preprocessed documents into the document store. You must add it at the end of your index.

1. Build your index and add DocumentWriter as its last component. You'll notice it requires a connection to a document store.

ℹ️

Index from a template

If you're using a template, DocumentWriter is already there connected to an OpenSearchDocumentStore. You can delete the document store and replace it with another one.

2. Choose a document store you want to use from the Document Stores group in the Component Library and drag it onto the canvas.
3. Connect the document store to DocumentWriter and configure it on the component card.
4. Save your index.

Retrieve Documents From the Document Store

Each document store has dedicated retrievers that are called the same as the document store. For example, MongoDBAtlasDocumentStore has two compatible retrievers: MongoDBAtlasEmbeddingRetriever and MongoDBAtlasFullTextRetriever.

Add the retrievers to your query pipeline and configure the document store they should connect to in the same way you configured DocumentWriter. Pass the secret's name as the API key for the document store.