fix(telemetry): include Anthropic cache tokens in total token extraction - #2573
Open
feiiiiii5 wants to merge 2 commits into
Open
fix(telemetry): include Anthropic cache tokens in total token extraction#2573feiiiiii5 wants to merge 2 commits into
feiiiiii5 wants to merge 2 commits into
Conversation
extract_token_usage() had no key names for Anthropic's cache_creation_input_tokens / cache_read_input_tokens fields, so total_tokens silently dropped cached tokens for Anthropic-backed calls. These tokens are billed by the provider and feed into backend high-usage anomaly checks via LLM_TOKENS_TOTAL, so under-reporting has operational impact beyond cost display. Extract both cache fields and add them to the computed total when no explicit total is provided; behavior is unchanged when the fields are absent or an explicit total_tokens is present. Fixes rhesis-ai#2572 Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com>
feiiiiii5
force-pushed
the
fix/anthropic-cache-token-extraction
branch
from
August 22, 2026 15:57
14f8dd0 to
aa7dcb2
Compare
![peqy[bot]](https://avatars.githubusercontent.com/in/1930020?s=60&v=4){kind=small}
There was a problem hiding this comment.
Improvement: total recomputation currently ignores the “cache-only” case (cache tokens present but input/output are 0), which would still under-report.
Everything else looks solid, and the added tests cover the main intended Anthropic/OpenAI/Gemini/None paths.
There was a problem hiding this comment.
Change looks good and tests cover the main Anthropic/OpenAI/Gemini cases.
Only remaining suggestion (see inline thread): consider computing total_tokens when only cache tokens are present (cache-hit scenarios with 0 input/output), e.g. widen the condition to include cache_creation_tokens / cache_read_tokens.
Widen the total-computation condition to include cache tokens per peqy review feedback: a provider returning only cache hits with zero input/output would previously report total=0. Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #2572
extract_token_usage()had no key names for Anthropic'scache_creation_input_tokens/cache_read_input_tokensfields, sototal_tokenssilently dropped cached tokens for any Anthropic-backed call. These tokens are billed by the provider and feed into backend high-usage anomaly checks viaLLM_TOKENS_TOTAL, so under-reporting has operational impact beyond cost display.Changes:
total_tokensis providedTests: Added
test_token_extraction.pycovering Anthropic with/without cache, OpenAI format, and None input. Verified manually: Anthropic usage{input:50, output:20, cache_creation:1000, cache_read:4000}now yields(50, 20, 5070)instead of(50, 20, 70).This contribution was developed with LLM assistance following repo conventions.