Release 2024.11
November brings new document stores, support for more models, optional indexing pipelines and more. Read on to find out!
More Document Stores
Document store is where your query pipelines access data. The indexing pipeline converts your files into documents and stores them in a document store of your choice. Until now, Deepset Cloud has supported the OpenSearch document store, but the list has grown. Starting with this release, you can use:
Each document store comes with dedicated Retrievers to make retrieval make the best use of the document store's technology.
Check Document Stores for more details.
Sample Datasets
Want to try a pipeline quickly but have no data? Use the sample files we prepared on healthcare, law, and tech and finance. They're available on the Files page:
Export Your Pipelines as Python
Save your pipelines locally as Python files. Pipelines that include deepset Cloud custom components (like our templates) may need additional adjustments and won't work out of the box after export. Export is available in Pipeline Builder - open any pipeline for editing and you'll find it there.
Support For More Models and Integrations
Secrets
Create secrets to securely manage connections to model providers or integrations. For more information, see Add Secrets to Connect to Third Party Providers.
We've added support for the following model providers:
- Fastembed: Models for generating embeddings and ranking documents. We've added the following components through which you can use Fastembed models:
- Google Vertex AI available through the following Generators:
- Jina AI embedding and ranking models available through:
- Embedders:
- Rankers:
- Llama.cpp library for efficient inference of large language models, available through:
- Generators:
- Mistral AI models available through an API with a pay-as-you-go access to the latest Mistral models. You can use it through:
- Embedders:
- Generators:
- Ollama models available through:
- Embedders:
- Generators:
And the following integrations:
- Langfuse available through LangfuseConnector
- Ragas available through RagasEvaluator
Naming Changes
To make things clearer and better reflect what they’re for, we’ve updated a couple of names:
- Studio is now Pipeline Builder
- Prompt Studio is now Prompt Explorer
The names have changed, but the functionality remains the same.
Indexing Pipelines No Longer Required
The range of scenarios deepset Cloud covers is growing, so we're adjusting to make workflows smooth and convenient. That's why indexing pipelines are no longer mandatory. You can have a pipeline with just the query part, and that's fine. This should make use cases like summarization or querying data in an external database, like Snowflake, smoother.
For other use cases where you query data in one of the document stores integrated with deepset Cloud, we still recommend using an indexing pipeline to prepare your data.
Labelling and V1 Templates Are Going Away
Having analyzed the data, we've decided to deprecate the Labelling feature. You can use Feedback to label your document search pipelines.
We're also deprecating version 1 templates as we prepare to fully switch to version 2. Your version 1 pipelines will continue to work, but you won't be able to create them.
Updated 21 days ago