Skip to main content

Use Cohere Models

Use Cohere models through Cohere API in your pipelines.

You can use Cohere's embedding and ranking models as well as LLMs. For a full list of supported models, see Cohere documentation.

Prerequisites​

You need an active Cohere API key to use Cohere's models.

Use Cohere Models​

First, connect deepset AI Platform to Cohere through the Integrations page. You can set up the connection for a single workspace or for the whole organization:

Add Workspace-Level Integration​

  1. Click your profile icon and choose Settings.
  2. Go to Workspace>Integrations.
  3. Find the provider you want to connect and click Connect next to them.
  4. Enter the API key and any other required details.
  5. Click Connect. You can use this integration in pipelines and indexes in the current workspace.

Add Organization-Level Integration​

  1. Click your profile icon and choose Settings.
  2. Go to Organization>Integrations.
  3. Find the provider you want to connect and click Connect next to them.
  4. Enter the API key and any other required details.
  5. Click Connect. You can use this integration in pipelines and indexes in all workspaces in the current organization.

Then, add a component that uses a Cohere model to your pipeline. Here are the components by the model type they use:

  • Embedding models:

    • CohereTextEmbedder: Calculates embeddings for text, like query. Often used in query pipelines to embed a query and pass the embedding to an embedding retriever.

    • CohereDocumentEmbedder: Calculates embeddings for documents. Often used in indexes to embed documents and pass them to DocumentWriter.

      Embedding Models in Query Pipelines and Indexes

      The embedding model you use to embed documents in your indexing pipeline must be the same as the embedding model you use to embed the query in your query pipeline.

      This means the embedders for your indexing and query pipelines must match. For example, if you use CohereDocumentEmbedder to embed your documents, you should use CohereTextEmbedder with the same model to embed your queries.

  • LLMs:

    • CohereGenerator: Generates text using a Cohere model, often used in RAG pipelines.

  • Ranking models:

    • CohereRanker: Ranks documents based on their similarity to the query. Often used in query pipelines to rank documents from the retriever and pass them on to PromptBuilder.

Usage Examples​

This is an example of how to use Cohere's embedding and ranking models and an LLM in an index and a query pipeline (each in a separate tab):

components:
  ...
    splitter:
      type: haystack.components.preprocessors.document_splitter.DocumentSplitter
      init_parameters:
        split_by: word
        split_length: 250
        split_overlap: 30

    document_embedder:
      type: haystack_integrations.components.embedders.cohere.document_embedder.CohereDocumentEmbedder
      init_parameters: 
        model: embed-english-v3.0

    writer:
      type: haystack.components.writers.document_writer.DocumentWriter
      init_parameters:
        document_store:
          type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
          init_parameters:
            embedding_dim: 768
            similarity: cosine
        policy: OVERWRITE
        
connections:  # Defines how the components are connected
  ...
  - sender: splitter.documents
    receiver: document_embedder.documents
  - sender: document_embedder.documents
    receiver: writer.documents

This is where you pass the model in these components in Pipeline Builder:

The component cards of components using Cohere models with the model parameter highlighted.