feat: populate usage telemetry in PromptResponse and UsageUpdate#166
Closed
simonrosenberg wants to merge 2 commits intozed-industries:mainfrom
Closed
feat: populate usage telemetry in PromptResponse and UsageUpdate#166simonrosenberg wants to merge 2 commits intozed-industries:mainfrom
simonrosenberg wants to merge 2 commits intozed-industries:mainfrom
Conversation
…fications The ACP protocol defines `PromptResponse.usage` and `UsageUpdate` session notifications (gated behind the `unstable_session_usage` feature), but they were never populated. Clients relying on these fields for benchmarking cost/token tracking received empty data. This change: - Handles `EventMsg::TokenCount` events instead of ignoring them - Extracts `TokenUsageInfo` (total token counts, context window) and builds an ACP `Usage` object with input/output/cached/reasoning tokens - Sends `UsageUpdate` session notifications when token counts arrive - Returns the `Usage` object in the `PromptResponse` at turn completion Resolves zed-industries#165 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tive totals UsageUpdate.used should represent "tokens currently in context" — the actual context window utilization. The previous implementation passed total_tokens (cumulative session total of input+output+cached+reasoning) which has two problems: 1. It was cumulative across all turns, so after N turns it showed N*X tokens even though the actual context was only X tokens 2. It included output and reasoning tokens, which are the model's response, not context window consumption — this could make used exceed size (e.g., 210K used / 200K size) which is misleading Now uses last_token_usage (per-turn) with only input + cached_input tokens, which reflects the actual context sent to the model. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Author
|
Closing in favor of #167 by @josevalim — his approach is cleaner: uses the canonical |
Contributor
|
Oops, I didn't see there was already an open PR. In any case, I do think mine is clearer for the usage bits but it doesn't do the token bits. Perhaps you may want to submit that as a PR once mine is merged! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
EventMsg::TokenCountevents (previously ignored with a "TODO" comment) to extract token usage data from the Codex engineUsageUpdatesession notifications with cumulative token counts and context window sizeUsageinPromptResponse(input tokens, output tokens, cached input tokens, reasoning tokens, total tokens)unstable_session_usagefeature flag inagent-client-protocol, which codex-acp already enables viafeatures = ["unstable"]Details
The Codex engine emits
EventMsg::TokenCountevents containingTokenUsageInfowith:total_token_usage: TokenUsage— cumulative session totalslast_token_usage: TokenUsage— per-turn countsmodel_context_window: Option<i64>— context window sizeThese events were being ignored with a comment: "In the future we can use this to update usage stats". This PR implements that future:
TokenCounthandler extractstotal_token_usageand builds an ACPUsageobject, stores it inPromptState.last_usageUsageUpdatenotification is sent viaSessionUpdate::UsageUpdatewith token totals and context windowTurnCompletehandler passeslast_usageback through the response channel alongsideStopReasonCodexAgent::prompt()attaches theUsageto thePromptResponsevia.usage()Test plan
(StopReason, Option<Usage>)return typecargo build— requires Rust toolchain (not available in dev environment, but types and patterns follow existing code exactly)cargo test— existing tests should pass unchanged (usage isNonein mock scenarios)Resolves #165
🤖 Generated with Claude Code