Update a Pipeline

Improve your pipeline to make your search faster and more accurate. Updating pipelines is done in a YAML file by default, but you can also open it in a Jupyter Notebook. You cannot change the pipeline name.

📘

You must be an Admin user to perform this task.

Update a Pipeline Using YAML

This is the default way of updating pipelines. You can use it if you want to quickly replace one node with another or update the parameters.

  1. Log in to deepset Cloud and go to Pipelines.
  2. Click the pipeline that you want to update.
  3. Modify the YAML file and save it.
  4. Deploy the pipeline to use it for search.

Update a Pipeline Using a Jupyter Notebook

This is a good method if you need to troubleshoot a pipeline. You can then easily share your work by dragging the notebook to your organization's shared folder in the Jupyter lab. All members of your organization will be able to work with it.

  1. Log in to deepset Cloud and go to Pipelines.
  2. Click the name of the pipeline that you want to modify. It opens in a YAML editor.
  3. Click Open in notebook and choose the type of the server that you want to use. (If you already have a server started, you're redirected to it without having to select again.) You can choose one of these server types:
    • CPU - use this server if you want to work with your pipelines in JupyterLab without running them
    • GPU - use this server if you want to run your pipelines in JupyterLab
  4. Wait until the server starts. It may take a couple of minutes.
  5. When the server's ready, open the notebook.
    A generated Jupyter notebook with your pipeline as code opens.

📘

Under the hood

Here's what happened: We loaded your pipeline from deepset Cloud and generated the code by running pipeline.to_notebook_cell. You can do this yourself for other pipelines, using the SDK.

  1. You can now work with your pipeline. When you're done, use the save_to_deepset_cloud method to save your pipeline. To make things easier for you, we already added this code at the bottom of your generated notebook.

Update a Pipeline with REST API

🚧

Coming soon

With this method, you can quickly upload an updated pipeline YAML to deepset Cloud. You need to Generate an API Key first. Run:

curl --request POST \
     --url https://api.cloud.deepset.ai/api/v1/workspaces/workspace_name/pipelines \
     --header 'Accept: application/json' \
     --header 'Authorization: Bearer <your_API_key>'

You can find more information about this endpoint at create pipeline API


Related Links