Synchronize Your Data

Data keeps changing and you want your search to always run on the latest data. Here are some tips how to do it.

To find an answer to a query, your pipeline searches the files you uploaded to deepset Cloud. When you deploy the pipeline, the files are indexed and ready to be searched. However, data is rarely static, and it's only natural that your files should change: get deleted, added, or updated. Your app should always be in sync with your file storage.

Syncing Your Data

The easiest way to ensure your search runs on the latest version of your files is to create a script that periodically syncs your data with deepset Cloud. deepset Cloud provides endpoints for uploading and deleting files so you can easily remove old data and add new ones.


Coming Soon: Updating Files

Currently, there is no endpoint for updating files, so if a file that's already in deepset Cloud is modified, you must delete it and upload the latest version.

Another option is to use the deepset Cloud SDK and upload files with the overwrite option. The SDK currently has no option to delete files, so you'd need to do that through API endpoint or the UI.


Whenever you add a file, it's automatically re-indexed. You don't have to redeploy the pipeline to trigger indexing.