Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Monitor Model Costs and Token Usage

To keep track of your model spend and find out which workspaces, services, pipelines, providers, and models drive it, use the Cost Management page. It shows how much your model calls cost and how many tokens they use.

About This Task​

The Cost Management page shows costs for your whole organization or, depending on your role, for a workspace. Costs are estimates: we calculate them from the token usage recorded in your traces and the model's price. To check credits and document storage units instead, see Understand Your Pipeline Usage. To change a model's price, see Set Model Prices.

The page refreshes automatically every minute while it's open. Click Auto-refresh to refresh right away.

Permissions​

Organization Admins can see the cost data of the whole organization and of any workspace. Other users can open the page if their role has the Costs permission. They see the data of the workspace they're in and can't switch to another one. For details, see User Roles and Permissions.

How Days Are Counted​

The Cost Management page counts every day in the date range as a UTC day, the same way providers bill. When you pick a day, the page shows the spend from 00:00 to 24:00 UTC of that day, even if your browser is in another timezone. For example, if you pick October 16 in Berlin, you see the spend of October 16 UTC, not of October 16 in Berlin time.

This means the total for a day matches the same day in your provider's console or invoice, and matches the cost exports. To reconcile spend with a provider, compare the same UTC day on both sides. The daily spend chart places each bar on its UTC day, and the Projected this month figure is also calculated from UTC days.

Pipeline Debug Runs​

If your spend includes testing, note that the model calls of pipeline debug runs, such as runs you start from Builder, are billed by the provider, so the Cost Management page counts them too. They appear as query usage, the same as other queries. If the run uses a saved pipeline, the page groups the usage under that pipeline; a run of an unsaved pipeline configuration isn't linked to a pipeline. A failed debug run counts because its calls were billed. A run that stops at a breakpoint counts the calls made so far, and resuming adds only the calls made after the breakpoint, so no call is counted twice.

Open the Cost Management Page​

In Haystack Enterprise Platform, click Cost Management in the sidebar, or open the command palette and search for Cost Management. You can see the spend for the whole organization over the date range you choose. You can also choose the workspaces you want to see.

The page URL keeps the scope, date range, and filters, so to share your current view, copy the URL and send it.

Understand the Summary​

To check how much you've spent and where the month is heading, read the summary at the top of the page. It shows these figures for the scope and date range you selected:

  • Total spend: The cost of all priced calls in the range, with a comparison against the previous period of the same length. If there is no earlier data, the page says No prior period to compare yet. The total includes the cost of model calls made during indexing.
  • Requests: The number of LLM calls. Each indexing run counts as one request, in the summary and in the table rows.
  • Tokens: The number of tokens used by those calls.
  • Avg / request: The average cost of one call.
  • Avg / day: The average cost per day in the range.
  • Projected this month: The month-to-date spend plus the average daily spend of the last 7 days applied to the remaining days of the month. Use it to see where the month is heading.

Calls That Aren't in the Total​

If some calls use a model without a matching price, they can't be priced and stay out of the total. Calls that are still being priced are added automatically, usually within seconds.

To find out which models need a price, click Show unmatched models in the warning. The Unmatched models list shows each model with its number of calls and tokens, starting with the model that affects the most calls, so you can price that one first. If you're an Organization Admin, click a model name to set a price for it, which applies to all of its calls. For details, see Set Model Prices. Click Hide details to collapse the list.

Explore the Breakdown​

To compare spend across groups for the whole date range, use the Spend breakdown table below the Cost flow chart. It lists spend by the dimension you choose in its Group by control. The options depend on your scope:

  • Whole organization: Workspace, Service, Pipeline, Model, and Provider.
  • One workspace: Service, Pipeline, Model, and Provider.
  • One service or pipeline: Model and Provider. The table groups by provider by default.

To separate indexing cost from query cost or compare indexes, group by Source or Index. For details, see See the Cost of Indexing.

Each row shows:

  • Share of spend: The row's share of the total.
  • Requests: The number of calls.
  • Tokens: The number of tokens.
  • Avg / req: The average cost of one call.
  • Cost: The row's total cost.

Click a workspace, service, or pipeline row to scope the whole page to it. Click a provider row to filter the page by that provider.

  • To see which providers take the largest share of spend, check the By provider card.
  • To find your most expensive models, check the Top models card, which lists the five models with the highest cost.
  • To follow spend from your organization through workspaces and services to models, use the Cost flow chart. It shows which workspaces and services send the most spend to which models. Hover over a flow to see its cost.

Switch the Cost Flow Between Services and Pipelines​

A service and the pipeline it serves can look like two separate services in the same column. To avoid this, the Cost flow chart lists only one kind of query target at a time and sums up the other kind in a single node.

  • Choose Services to compare what each service costs. The chart shows one node for each service. All queries that ran directly agains pipelines go into one node named Deployed Pipelines.
  • Choose Pipelines to see the cost of queries that ran directly against pipelines. The chart shows one node for each pipeline, based on the queries that ran directly against it. All queries that ran through a service go into one node named Services.
  • Hover over a node to see its cost.

The chart lists nodes from most to least expensive. The node that sums up the other kind always comes last in the column, even when it's the most expensive one. A query made through a service counts as a service query in both views, even though it also records the pipeline it served.

See the Cost of Indexing​

To find out how much your indexes spend on models, check the indexing cost on the Cost Management page. Indexing cost covers the LLMs, Agents, and rankers that run inside your indexing pipelines. It's read from the indexing traces your indexes record. An index records a trace only when you capture one, so an index without traces shows no indexing cost. To capture one, see Capture an Indexing Trace with the REST API. Embedder usage isn't included, because it isn't counted as LLM spend.

  • In the Spend breakdown table, group by Source to compare query spend with indexing spend. Group by Index to compare the spend of each index.
  • In the Cost flow chart, each index appears as its own node next to your services or pipelines, in both the Services and the Pipelines view.
  • Indexing runs belong to no pipeline and no service, so their spend is included in the No pipeline and No service rows.

New indexing usage appears within a few minutes after an indexing batch finishes, so recent runs can take a little while to show. If an index still shows no cost after that, check that it records traces. See Capture an Indexing Trace with the REST API.

Cost is still counted for traces over 10 MB: they're stored without component inputs and outputs but keep the model and token counts of LLM, Agent, and ranker calls. Only a trace with so many calls that even the token counts don't fit under the limit loses them, and the cost of that batch can be undercounted.

See the Prices Behind Your Costs​

To find out which price each model's cost is based on, click Prices. The drawer shows the price in effect for the models in your current period and filters, and the default price of every model. It also lists the custom prices that apply in your workspace. Everyone who can open the page can view prices; only Organization Admins can change them. For details, see Set Model Prices.

Get Daily Cost Over the API​

To use daily cost data outside the Cost Management page, for example in your own reports, see Get Daily Cost. To find out which price each model's cost is based on, click Prices at the top of the page. This view shows the price in effect for the models in your current period and filters, and the default price of every model. It also lists the custom prices that apply in your workspace. Everyone who can open the page can view prices; only Organization Admins can change them. For details, see Set Model Prices.

Check the LLM Cost of a Service​

To check the recent LLM cost of a single deployed pipeline, go to Services, open the service, and click Overview. The LLM Cost (30d) card shows the total estimated cost and number of requests for that service over the last 30 days. The 30 days are UTC days, the way providers bill them, so the total matches the same days in your provider's console.