File Methods

Manage your files from your local environment using Python methods of the FileClient class. You can upload, list, and delete your files in deepset Cloud from your SDK.

Prerequisites

  • You must be an Admin to perform this task.
  • If you work from your local SDK, you must have Haystack installed. For more information, see Haystack Installation. If you use Jupyter Notebooks in deepset Cloud, you don't have to worry about that.
  • Add API endpoint and API key to the environment variables. The API endpoint is <https://api.cloud.deepset.ai/api/v1>. See Generate an API Key.

📘

Server restart

If the Notebooks server closes while you're still working, all the files you saved are still there. When you restart the server, you'll be able to work on them again.

Upload Files

Upload text or PDF files.

This table lists the parameters that you can use with the upload_files() method:

MethodParametersDescription
upload_files()file_paths - paths of txt or PDF files that you want to upload.

metas - metadata of the files to upload. A list of dictionaries. Optional.

workspace - the name of the deepset Cloud workspace to which you want to upload files.
Uploads files to deepset Cloud.
Example of usage
from haystack.utils import DeepsetCloud
from pathlib import Path

file_client = DeepsetCloud.get_file_client()

file_paths = [
  Path("C:\Users\OneDrive\Documents\file1.txt"),
  Path("C:\Users\OneDrive\Documents\file2.pdf")
]
metas = [{"file_id": "file1.txt"}, {"file_id": "file2.txt"}]
my_files = file_client.upload_files(file_paths=file_paths, metas=metas)

Delete Files

Delete files from deepset Cloud.

This table lists the parameters that you can use with the delete_file() method:

MethodParametersDescription
delete_file()file_id - The ID of the file that you want to delete. You can use list_files() to see all available files.
String. Required.

workspace - the name of the workspace where the file is.
String. Optional.
Deletes a file from a deepset Cloud workspace.
delete_all_files()workspace - the name of the workspace where the files are.
String. Optional.
Deletes all files from a deepset Cloud workspace.
Example of usage
from haystack.utils import DeepsetCloud

file_client = DeepsetCloud.get_file_client()

file_client.delete_file(file_id="12345566")

List Files

List all files that exist in a deepset Cloud workspace. You can filter the files by their names or metadata.

This table lists the parameters that you can use with the list_files() method:

MethodParametersDescription
list_files()name - the name or part of the name of the file that you want to list. String. Optional.

meta_key - the key of the metadata of the file that you want to filter for. String. Optional.

meta_value - the value of the metadata of the file that you want to filter for. String. Optional.

workspace - the name of the deepset Cloud workspace that contains the file. String. Optional.
Returns a Generator of all the files in a deepset Cloud workspace.
Example of usage
from haystack.utils import DeepsetCloud

file_client = DeepsetCloud.get_file_client()

my_files = [f for f in file_client.list_files()]
print(my_files)