Configure Index Settings
Control how an index runs: GPU acceleration, batch size, and queue length per replica.
About This Task
Most index settings can be changed only while the index is disabled, because an enabled index is read-only. Disable the index first, change the setting, and enable the index again. For details, see Edit an Index.
Other settings have their own guides:
- To run the index on a specific Haystack version, see Pin a Haystack Version for an Index.
- To index only some of your files, see Filter Files to Index.
- To choose the instance size of the indexing pod, see Set the Indexing Instance Size. You can change the size while the index is enabled.
Enable GPU Acceleration
By default, indexes run on CPU. You can enable GPU acceleration for an index to speed it up. This is especially useful for pipelines that include components that run faster on a GPU, like DoclingConverter or SentenceTransformersDocumentEmbedder.
When you enable GPU acceleration, the index checks whether a component needs a GPU and assigns one automatically. GPUs are only used when required and are not reserved for the entire index processing.
If GPU acceleration is disabled and your index includes components that rely on a GPU, those components run on the CPU instead. This can slow down processing and may cause timeouts, especially for larger or more complex indexes.
To enable GPU acceleration:
- Go to Indexes and click the index you want to enable GPU acceleration for. This opens the Builder.
- Open the Settings tab and click the GPU Acceleration toggle to turn it on.
If your index needs more CPU or memory rather than a GPU, for example to convert large files, set a larger indexing pod size.
Limit the Batch Size of an Index
An indexing pipeline processes files in batches. If an index handles heavy files, such as large documents that need OCR or LLM enrichment, a batch can take longer than the queue timeout allows, and the files get retried. Lower the maximum batch size to make batches smaller and reduce the chance of retries. If you leave the maximum batch size unset, the index uses the platform default.
The value must be between 1 and 500. You can change it while the index is enabled or disabled. On an enabled index, the new value applies the next time the indexing pod is deployed.
Set the Maximum Batch Size in the UI
- Log in to Haystack Enterprise Platform and open Indexes.
- Click the index name and open the index settings.
- In the Max allowed batch size section, enter a number from 1 to 500 in the Max batch size field. You need permission to edit indexes to change it.
- Press Enter or click outside the field to save the value.
To go back to the platform default, clear the field and click outside it.
Set the Maximum Batch Size with the API
Set the batch size with the max_allowed_batch_size parameter of the Update Index API endpoint. It accepts an integer from 1 to 500, or null to reset to the platform default. If you leave the parameter out of a request, the current value stays as it is. A value outside the range returns a 422 error. Send an integer from 1 to 500, or null.
The index response returns null when no value is set. For the full request and response schema, see the Update Index endpoint.
To find out whether batches are too large, check the throughput averages on the index's Metrics tab. See Read the Indexing Throughput Averages.
Set the Queue Length per Replica
An index adds replicas as its queue of files grows, up to the Max replicas you set. The queue length per replica is the number of queued files that one replica handles before another replica is added. Lower the value if a slow pipeline, for example one with OCR or LLM enrichment, takes too long on a small queue. A lower value scales out sooner, so more replicas start working on the same queue. The number of replicas never goes above Max replicas, so a low value has no effect if that limit is already reached.
The value must be a whole number from 1 to 1500. An empty field uses the default of 1500.
To set the queue length per replica:
- In Haystack Enterprise Platform, go to Indexes and click the index you want to configure. This opens the Index in Builder.
- Open the Settings tab.
- In the Queue Length per Replica section, type a value in the Queue length per replica field. The value is saved when you leave the field or press Enter.
To go back to the default, clear the field. The field then shows 1500 (default).
To confirm the value was saved, leave the field and reopen the Settings tab. The field shows your value.
What To Do Next
After you change the settings, enable the index so it applies them. Wait until indexing finishes before you run queries.
Related Information
Was this page helpful?