Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Get Daily Cost with the REST API

Get the LLM cost per day for your organization or a workspace, for example to use it in your own reports.

About This Task​

To use daily cost data outside the Cost Management page, call the Get Token Usage Cost Over Time endpoint, also called the cost-over-time endpoint. It returns the cost per UTC day in US dollars:

GET /api/v2/cost/organizations/{organization_id}/stats/cost-over-time

Users with the Costs permission must pass a workspace_id. Only Organization Admins can leave it out to get organization-wide data.

Prerequisites​

Get the Cost per Day​

Without extra parameters, the response contains one point per day. To split each day, add these optional query parameters:

  • group_by: The dimension to split by: model, workspace, pipeline, deployment, provider, or api_key. deployment is called a service in the UI. You can also split by source, which gives the groups query and indexing, or by index, which gives one group for each index.
  • group_limit: How many of the groups with the highest cost keep their own points. The default is 5 and the maximum is 20.

Totals include the cost of indexing runs, and each indexing run counts as one request. Because indexing runs belong to no pipeline and no service, their cost goes into the unattributed group when you split by pipeline or deployment.

This request splits each day by provider and keeps the five providers with the highest cost. Replace <ORGANIZATION_ID> with your organization ID and <YOUR_API_KEY> with your Haystack Platform API key:

curl --request GET \
  "https://api.cloud.deepset.ai/api/v2/cost/organizations/<ORGANIZATION_ID>/stats/cost-over-time?group_by=provider&group_limit=5" \
  --header "Authorization: Bearer <YOUR_API_KEY>"

Read the Response​

With group_by, the response contains:

  • group_by: The dimension you chose.
  • groups: Every group the points refer to, in stack order. Each group has a key, a label, and a kind. The kind is group for a group with high cost, other for the merged remaining groups, or unattributed for usage with no value for the dimension.
  • data: One point per day and group. Each point has a group field with one of the keys in groups. A day's points add up to the day's total.

Without group_by, group_by and groups are null.

Get the Cost per Conversation​

Without group_by, each point in data also has these fields, so you can chart cost per conversation:

  • conversations: The number of distinct conversations with usage that day. Each chat Session counts once, and a query outside a Session counts as its own conversation. Divide cost_total by conversations to get the day's average cost per conversation.
  • is_cost_complete: false when some of the day's generations are still waiting to be priced or have no known price. In that case, cost_total is a lower bound, so the average cost per conversation is too.

With group_by, both fields are null.

Keep these limits in mind when you read conversations:

  • A conversation that runs past midnight UTC counts on both days, and each day includes only its own cost.
  • Usage recorded before conversations were tracked counts as one conversation per query. On those days, conversations overstates the number of multi-turn conversations, so the average cost per conversation is understated.

Get Totals and Breakdowns​

The API also has endpoints for totals, breakdown, and cost flow. The breakdown endpoint returns one page of groups at a time. Use sort_by, sort_order, limit, and page_number to sort the groups and page through them, and check has_more to see whether more pages follow.