Filter Files to Index with the REST API
Choose which files an index processes by setting a file name prefix filter
About This Task
You set the filters with the Update Index endpoint. You can update the filters only while the index is disabled. Changing a filter doesn't re-index files that were already indexed. For more background, see Choose Files to Index.
Prerequisites
- You need an API key. For details, see Generate API Keys.
- You need a disabled index.
- To filter by metadata, your files need metadata. For details, see Add Metadata to Files with the REST API.
Set a File Name Prefix Filter
Pass a comma-separated list of prefixes in file_name_prefix_filter. A file is indexed if its name starts with any of the prefixes. The trailing * is optional: docs/ and docs/* both match names that start with docs/.
To index only files whose names start with legal or contracts/, send this request:
curl --request PATCH \
--url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/indexes/<INDEX_NAME> \
--header 'accept: application/json' \
--header 'authorization: Bearer <YOUR_API_KEY>' \
--header 'content-type: application/json' \
--data '
{
"file_name_prefix_filter": "legal*,contracts/"
}
'
Clear the File Name Prefix Filter
To remove the prefix filter, set file_name_prefix_filter to null:
curl --request PATCH \
--url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/indexes/<INDEX_NAME> \
--header 'accept: application/json' \
--header 'authorization: Bearer <YOUR_API_KEY>' \
--header 'content-type: application/json' \
--data '{"file_name_prefix_filter": null}'
Set a Metadata Filter
Each condition in file_meta_filter must include:
field: The metadata field name to filter on.operator: The comparison operator:==(equal) or!=(not equal).value: The value to compare against. Supported types: string, integer, number, boolean, or null.
All conditions are combined with AND logic: a file must satisfy every condition to be indexed. The filter applies to file metadata, not to file content.
To index only files in the "legal" category that are not in German, send this request:
curl --request PATCH \
--url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/indexes/<INDEX_NAME> \
--header 'accept: application/json' \
--header 'authorization: Bearer <YOUR_API_KEY>' \
--header 'content-type: application/json' \
--data '
{
"file_meta_filter": [
{"field": "category", "operator": "==", "value": "legal"},
{"field": "language", "operator": "!=", "value": "de"}
]
}
'
To remove a metadata filter, set file_meta_filter to null. You can set file_name_prefix_filter and file_meta_filter in the same request. When both are set, a file must match both filters to be indexed.
Related Information
Was this page helpful?