About This Task

To use your component in deepset Cloud, you must upload it first. There are three ways to upload your component:

• By creating a release of your forked repository (recommended if multiple users collaborate on the repository, lets you version control your code)
• Through REST API
• Using commands (currently supported for Linux and macOS)

When you upload a component, we verify the structure and version of the uploaded ZIP file.

Prerequisites

• Make sure you test your component before uploading.
• Make sure your component code follows appropriate linting and formatting conventions. You can use the hatch run code-quality:all command to check the linting.

Upload Your Component

Upload by Creating a Release

We use GitHub actions to build and import custom components to deepset Cloud. Before importing the component, the action runs the tests and checks the code quality. Only if those pass, the component is imported.

The action is triggered when you create a tag for a release in your GitHub repository. For details on tags and releases, see GitHub documentation.

1. Before you start importing, make sure you pushed all your changes to the forked repository.
2. Add the DEEPSET_CLOUD_API_KEY secret to your repository:
   1. Go to Settings > Secrets and variables > Actions.
   2. Click New repository secret.
   3. Type DEEPSET_CLOUD_API_KEY as the secret name, paste the deepset Cloud API key in the Secret field, and click Add secret.
3. Make sure your repository has workflows enabled: Go to Actions and click Enable workflows. (If they're enabled, you won't see this option.)
4. Create a new release:
   1. In the left-hand navigation, click Releases and choose Draft a new release.
   2. Create a tag for your release.
      Note: The version you specified in the __about__ file will be overwritten by the tag value. Make sure the tag matches the version number.
   3. Publish your release.

This starts the tests and code quality check. You can monitor the action progress in the Actions tab of your repository. Once all the tests and checks finish successfully, your component is imported to deepset Cloud.

Upload with REST API

1. Zip the repository from the template folder. The zipped repository should contain the same files in the same hierarchy as the dc-custom-component-template repository.
   This command creates a ZIP file called custom_component.zip in the parent directory:

   Linux and macOSWindows

   zip -r ../custom_component.zip ./*
   

   Compress-Archive -Path .\* -DestinationPath ..\custom_component.zip -Force
   

2. Upload the ZIP file to deepset Cloud using the Import Custom Components endpoint. Here's a sample code you can use as a starting point for your request:

   cURLPython

   curl 
   --request POST \
   --url https://api.cloud.deepset.ai/api/v2/custom_components \
   --header 'accept: application/json' \
   --header 'Authorization: Bearer api_XXX' \
   --form 'file=@"/path/to/custom/component/custom_component.zip";type=application/zip'
   

   import requests
   
   url = "https://api.cloud.deepset.ai/api/v2/custom_components"
   
   files = { "file": ("custom_component.zip", open("path/to/custom_component.zip", "rb"), "application/x-zip-compressed") }
   headers = {
       "accept": "application/json",
       "authorization": "Bearer deepset_cloud_API_key"
   }
   
   response = requests.post(url, files=files, headers=headers)
   
   print(response.text)
   

Upload with Commands

This method is currently available for Linux and macOS systems.

1. Set your deepset Cloud API key:
   Shell

   export API_KEY=<YOUR_API_KEY>
   

2. From within this project (custom component template),run the following command to upload your custom component:
   Shell

   hatch run dc:build-and-push
   

This command creates a ZIP file called custom_component.zip in the dist directory and uploads it to deepset Cloud.

Verify the Upload

Check the component status using the Get Custom Components endpoint. If the status is finished, your component is ready for use in your pipelines. Here is a sample code you can use:

import requests

url = "https://api.cloud.deepset.ai/api/v2/custom_components"

headers = {
    "accept": "application/json",
    "authorization": "Bearer deepset_cloud_API_key"
}

response = requests.get(url, headers=headers)

print(response.text)

curl --request GET \
     --url https://api.cloud.deepset.ai/api/v2/custom_components \
     --header 'accept: application/json' \
     --header 'authorization: Bearer deepset_cloud_API_key'

ℹ️

It takes a while for the custom component to be available in Pipeline Builder. Refresh the page and wait a couple of minutes.

What to Do Next

Once the component is successfully uploaded to deepset Cloud, you can use it in your pipelines. The component will be visible in Pipeline Builder's components library. (To access Pipeline Builder, find your pipeline on the Pipelines page and choose Edit from the More Actions menu next to it.)

In the component library, your component is in a group whose name is the same as the name of the template folder where you saved your component. For example, if you component code is in ./dc-custom-component-template/src/dc_custom_component/rankers/my_ranker.py, you'll find it in the Rankers group.