Jump to Content
deepset Cloud
GuidesAPI and SDK Reference
RegisterLog indeepset Cloud
Guides
RegisterLog in
GuidesAPI and SDK Reference

Getting Started

  • What's deepset AI Platform?
    • Feature List
    • deepset AI Platform and Haystack
  • deepset AI Platform Status
  • What's New
    • deepset AI Platform 2.0
    • Release 2025.4
    • Release 2025.3
    • Release 2025.2
    • Release 2025.1
    • Release 2024.12
    • Release 2024.11
    • Release 2024.9
    • Release 2024.8
    • Release 2024.7
  • Quick Start Guide
  • Working in deepset AI Platform
  • Basic Concepts
  • Third-Party Software

Tutorials

  • Learn the Basics
    • Tutorial: Building Your First Question Answering App
    • Tutorial: Building Your First Document Search App
    • Tutorial: Building a Robust RAG System
    • Tutorial: Integrating deepset API with Your Frontend App
    • Tutorial: Uploading Files with Metadata through SDK CLI
  • Learn More Advanced Features
    • Tutorial: Uploading Files with Python Methods
    • Tutorial: Demoing Your App
    • Tutorial: Creating a Custom RegexBooster Component
  • REST API Tutorials
    • Tutorial: Creating a Chat App Through REST API
    • Tutorial: Managing Feedback Entries Through REST API

How-To Guides

  • Working with Your Data
    • Upload Files
    • Working with Metadata
      • Add Metadata to Your Files
      • Use Metadata in Your Search System
      • Filter Syntax
    • Connect to Your Snowflake Database
    • Connect to an External Document Store
    • Synchronize Data Using Airbyte
    • Setting Up Your VPC
      • Connect Your OpenSearch Cluster
      • Connect Your S3 Bucket
  • Working with Indexes
    • Create an Index
    • Edit an Index
    • Enable an Index
    • PreProcessing Data with Pipeline Components
    • Use Unstructured to Process Documents
    • Use Azure Document Intelligence
  • Designing Your Pipeline
    • Create a Pipeline
      • Create a Pipeline in Pipeline Builder
      • Create a Pipeline with REST API
    • Import a Pipeline
    • Working with Custom Components
      • Create a Custom Component
      • Upload a Custom Component to deepset AI Platform
      • Update a Custom Component
      • Troubleshoot Custom Components
    • Using Hosted Models and External Services
      • Use Amazon Bedrock and SageMaker Models
      • Use Azure OpenAI Models
      • Use Cohere Models
      • Use DeepL Translation Services
      • Use Google Gemini Models
      • Use Google Search API
      • Use Hugging Face Models
      • Use NVIDIA Models
      • Use OpenAI Models
      • Use Together AI Models
      • Use Voyage AI Models
    • Edit a Pipeline
    • Building with Large Language Models (LLMs)
      • Using Prompt Explorer
      • Writing Prompts in deepset AI Platform
      • Enable References for Generated Answers
      • Enable Streaming
      • Set Up Tool Calling For Your Model
    • Settting Additional Pipeline Options
      • Change the Pipeline's Service Level
      • Set Pipeline Output Type
      • Run a Pipeline in Async Mode
    • Deploy a Pipeline
    • Troubleshoot Pipelines and Indexes
  • Evaluating Your Pipeline
    • Share a Pipeline Prototype
    • Collect User Feedback
    • Guidelines for Onboarding Your Users
    • Export Users' Feedback
  • Optimizing Your Pipeline
    • Improving Your Document Search Pipeline
    • Boosting Retrieval with OpenSearch Queries
    • Improving Your Question Answering Pipeline
  • Testing Your Pipeline
    • Test Your Pipeline in Playground
    • Set Default Pipeline
    • Modify Pipeline Parameters at Query Time
    • Debug Your Query Pipelines Remotely
  • Productionizing Your Pipeline
    • Use Your Pipeline in Your Target App
    • Monitor Pipeline Performance
    • Check the Groundedness Score
    • Understand Your Pipeline Usage
    • Implement CI/CD with Github Actions
    • Trace Your Pipelines
      • Trace with Weights & Biases Weave
      • Trace with Langfuse
  • Working with the SDK
    • Install the SDK
    • Using the Command-Line Interface (CLI)
      • Set Up the SDK CLI
      • Upload Files with CLI
      • Download Files with CLI
      • List Existing Files with CLI
    • Using Python Methods
      • Upload Files with Python
      • List Existing Files with Python
      • Get Upload Sessions Details
      • Download Files with Python
  • Working with Jobs
    • Prepare a Query Set
    • Create a Job
    • Share a Job
  • Managing Access
    • Manage User Access
    • Connect to Model and Service Providers
    • Generate an API Key
    • Add Secrets to Connect to Third Party Providers

Concepts

  • Indexes
    • PreProcessing Data with deepset Components
  • Pipelines
  • Pipeline Examples
    • Auto Loan Payment Chatbot
  • AI Agents
    • Agentic Pipelines
    • The Agent Component
  • Document Stores
    • ElasticsearchDocumentStore
    • MongoDBAtlasDocumentStore
    • OpenSearchDocumentStore
    • PineconeDocumentStore
    • QdrantDocumentStore
    • WeaviateDocumentStore
  • Pipeline Components
    • Custom Components
    • Common Component Combinations
    • Advanced Component Connections
    • Agents
      • Agent
    • Audio
      • LocalWhisperTranscriber
      • RemoteWhisperTranscriber
    • Augmenters
      • DeepsetAnswerBuilder
      • DeepsetFileDownloader
      • DeepsetFileUploader
      • DeepsetMetaFieldDocumentGrouper
      • DeepsetOpenSearchFilterParser
      • DeepsetStaticAnswerBuilder
      • ReferencePredictor
    • Auth
      • CloudSQLAuthProxy
    • Builders
      • AnswerBuilder
      • ChatPromptBuilder
      • PromptBuilder
    • Classifiers
      • DocumentLanguageClassifier
      • TransformersZeroShotDocumentClassifier
    • Caching
      • CacheChecker
    • Connectors
      • JinaReaderConnector
      • LangfuseConnector
      • OpenAPIConnector
      • OpenAPIServiceConnector
      • WeaveConnector
    • Converters
      • AzureOCRDocumentConverter
      • CSVToDocument
      • DeepsetCSVRowsToDocumentsConverter
      • DeepsetDeepLDocumentTranslator
      • DeepsetDeepLTextTranslator
      • DeepsetFileToBase64Image
      • DeepsetPDFDocumentToBase64Image
      • DeepsetSourceToBase64Image
      • DeepsetVLMPDFToDocumentConverter
      • DOCXToDocument
      • HTMLToDocument
      • JSONConverter
      • MarkdownToDocument Converter
      • MsgToDocument
      • MultiFileConverter
      • OpenAPIServiceToFunctions
      • OutputAdapter
      • PDFMinerToDocument
      • PPTXToDocument
      • PyPDFToDocument
      • TextFileToDocument
      • TikaDocumentConverter
      • UnstructuredFileConverter
      • XLSXToDocument
    • Crawlers
      • DeepsetFirecrawlWebScraper
    • DocumentWriter
    • Embedders
      • AmazonBedrockDocumentEmbedder
      • AmazonBedrockTextEmbedder
      • AzureOpenAIDocumentEmbedder
      • AzureOpenAITextEmbedder
      • CohereDocumentEmbedder
      • CohereTextEmbedder
      • DeepsetNvidiaDocumentEmbedder
      • DeepsetNvidiaNIMDocumentEmbedder
      • DeepsetNvidiaNIMTextEmbedder
      • DeepsetNvidiaTextEmbedder
      • FastembedDocumentEmbedder
      • FastembedSparseDocumentEmbedder
      • FastembedSparseTextEmbedder
      • FastembedTextEmbedder
      • HuggingFaceAPIDocumentEmbedder
      • HuggingFaceAPITextEmbedder
      • JinaDocumentEmbedder
      • JinaTextEmbedder
      • MistralDocumentEmbedder
      • MistralTextEmbedder
      • NvidiaDocumentEmbedder
      • NvidiaTextEmbedder
      • OllamaDocumentEmbedder
      • OllamaTextEmbedder
      • OpenAIDocumentEmbedder
      • OpenAITextEmbedder
      • SentenceTransformersDocumentEmbedder
      • SentenceTransformersTextEmbedder
      • VoyageDocumentEmbedder
      • VoyageTextEmbedder
    • Executors
      • DeepsetParallelExecutor
    • Extractors
      • ExtractiveReader
      • NamedEntityExtractor
      • LLMMetadataExtractor
    • Fetchers
      • LinkContentFetcher
    • Generators
      • Generators Overview
      • AmazonBedrockChatGenerator
      • AmazonBedrockGenerator
      • AnthropicChatGenerator
      • AnthropicGenerator
      • AnthropicVertexChatGenerator
      • AzureOpenAIChatGenerator
      • AzureOpenAIGenerator
      • CohereChatGenerator
      • CohereGenerator
      • DALLEImageGenerator
      • DeepsetAmazonBedrockGenerator
      • DeepsetAmazonBedrockVisionGenerator
      • DeepsetAzureOpenAIVisionGenerator
      • DeepsetTogetherAIGenerator
      • GoogleAIGeminiChatGenerator
      • GoogleAIGeminiGenerator
      • HuggingFaceAPIChatGenerator
      • HuggingFaceAPIGenerator
      • HuggingFaceLocalChatGenerator
      • HuggingFaceLocalGenerator
      • LlamaCppChatGenerator
      • LlamaCppGenerator
      • MistralChatGenerator
      • NvidiaGenerator
      • OllamaChatGenerator
      • OllamaGenerator
      • OpenAIChatGenerator
      • OpenAIGenerator
      • SagemakerGenerator
      • VertexAICodeGenerator
      • VertexAIGeminiChatGenerator
      • VertexAIGeminiGenerator
      • VertexAIImageCaptioner
      • VertexAIImageGenerator
      • VertexAIImageQA
      • VertexAITextGenerator
    • Inputs
      • FilesInput
      • Filters
      • Query
    • Joiners
      • AnswerJoiner
      • BranchJoiner
      • DocumentJoiner
      • ListJoiner
      • StringJoiner
    • Outputs
      • Output
    • PreProcessors
      • CSVDocumentCleaner
      • CSVDocumentSplitter
      • DocumentCleaner
      • DocumentPreprocessor
      • DocumentSplitter
      • HierarchicalDocumentSplitter
      • RecursiveDocumentSplitter
      • TextCleaner
    • Rankers
      • CohereRanker
      • DeepsetNvidiaRanker
      • FastembedRanker
      • JinaRanker
      • LostInTheMiddleRanker
      • MetaFieldGroupingRanker
      • MetaFieldRanker
      • NvidiaRanker
      • SentenceTransfomersDiversityRanker
      • TransformersSimilarityRanker
    • Retrievers
      • ElasticsearchBM25Retriever
      • ElasticsearchEmbeddingRetriever
      • FilterRetriever
      • MongoDBAtlasEmbeddingRetriever
      • MongoDBAtlasFullTextRetriever
      • OpenSearchBM25Retriever
      • OpenSearchEmbeddingRetriever
      • PineconeEmbeddingRetriever
      • QdrantEmbeddingRetriever
      • QdrantHybridRetriever
      • QdrantSparseEmbeddingRetriever
      • SentenceWindowRetrier
      • SnowflakeTableRetriever
      • WeaviateBM25Retriever
      • WeaviateEmbeddingRetriever
    • Routers
      • ConditionalRouter
      • FileTypeRouter
      • MetadataRouter
      • TextLanguageRouter
      • TransformersTextRouter
      • TransformersZeroShotTextRouter
    • Samplers
      • TopPSampler
    • SuperComponents
      • SuperComponent
    • Tools
      • ToolInvoker
    • Web Search
      • SearchApiWebSearch
      • SerperDevWebSearch
  • Jobs
  • Data Flow in deepset AI Platform
  • Language Models in deepset AI Platform

Learn

  • Building an AI-Based App
    • Components of an AI-Based App
    • 5-Step Guide to Building a Successful Prototype
  • Retrieval Augmented Generation (RAG) Question Answering
  • Document Search
  • Prompt Engineering Guidelines
  • Extractive Question Answering
  • Language Models
    • Large Language Models Overview

Setting Up Your VPC

Connect your virtual storage to the deepset AI Platform to stay in full control of your data.

Suggest Edits

Connect your OpenSearch cluster

Connect your S3 bucket

Updated 3 months ago


  • Open Source
  • Haystack
  • Resources
  • Models
  • Datasets
  • FARM
  • Company
  • Blog
  • About
  • Jobs

Building a semantic layer for the modern tech stack — driven by the latest NLP and open source.

  • Privacy
  • Imprint
© 2024 deepset GmbH. All rights reserved.