Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Why Is There a Spike in Indexing Tasks?

An increased number of indexing tasks for an index may be caused by metadata updates. Each metadata change on a file in Haystack Enterprise Platform queues work on the index. A large batch of edits produces a large batch of tasks. That is expected behavior, not a fault in the system.

What You Might Notice​

  • The Indexes page shows more pending or running tasks right after you change metadata on files.
  • A bulk metadata update across many documents causes a sharp rise in the task count.

How It Works​

  • Changing metadata on a single document creates one indexing task for that document.
  • Applying the same metadata change across many documents creates one task per affected document.

The index pipeline picks up these tasks and applies them in order. While work is still in the queue, the task count stays higher than usual.

What to Expect​

Tasks finish as the index catches up. The pending task count drops on its own when processing completes. You don't need to take action unless something stays stuck for an unusually long time and you have ruled out a large metadata change as the cause.

Tips​

If the task count spikes and you are unsure why, check whether a script, integration, or teammate recently ran a bulk metadata update before you open a support ticket.