Enhancement: Token Usage & Cost Estimator#8759
Enhancement: Token Usage & Cost Estimator#8759abhinav2105 wants to merge 6 commits intoaaif-goose:mainfrom
Conversation
Feat: Add session summarizer enhancement.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a8093a814a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| model_key = DEFAULT_MODEL | ||
| for k in pricing: | ||
| if k != "default" and k.lower() in (provider or "").lower(): | ||
| model_key = k |
There was a problem hiding this comment.
The per-session pricing logic matches pricing keys against provider_name, but Goose stores provider IDs like openai/anthropic/tetrate in that column while your pricing table is keyed by model IDs like gpt-4o and claude-haiku-4-5. In practice this loop usually misses and falls back to DEFAULT_MODEL, so many sessions are priced at the wrong rate (for example, OpenAI sessions being billed as Claude Haiku).
Useful? React with 👍 / 👎.
|
|
||
| total_input = sum(s["input_tokens"] or 0 for s in sessions) | ||
| total_output = sum(s["output_tokens"] or 0 for s in sessions) | ||
| _, _, total_cost = calculate_cost(total_input, total_output, pricing, DEFAULT_MODEL) |
There was a problem hiding this comment.
The markdown report's cumulative total_cost is computed by applying DEFAULT_MODEL pricing to aggregated tokens, which ignores the per-session provider/model rates used in the table above. When sessions use mixed providers/models, the reported cumulative cost will diverge from the sum of per-session estimates and misstate spend.
Useful? React with 👍 / 👎.
| ORDER BY created_timestamp ASC | ||
| """, |
There was a problem hiding this comment.
get_messages orders only by created_timestamp, but Goose message timestamps are coarse enough that multiple rows can share the same value in a single turn. SQLite does not guarantee row order for ties, so transcript chunks can be shuffled and produce incorrect summaries. Add a stable secondary sort key (for example, message row id) to keep chronological order deterministic.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4aeef6ec53
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| for k in pricing: | ||
| if k != "default" and k.lower() in provider_lower: | ||
| return k |
There was a problem hiding this comment.
resolve_model_key returns the first pricing key found as a substring of provider, which can select a broader model before a more specific one. With the current key order, a string containing gpt-4o-mini matches gpt-4o first, so those sessions are priced at the higher gpt-4o rates. This produces incorrect cost estimates whenever model IDs share prefixes; matching exact keys first (or longest keys first) avoids that drift.
Useful? React with 👍 / 👎.
| elif "openai" in p or "azure" in p or "gpt" in p: | ||
| base = "https://api.openai.com/v1" if "openai" in p else None | ||
| return summarize_with_openai_compat( |
There was a problem hiding this comment.
The azure branch is routed through the generic OpenAI client with base_url=None, which sends requests to the default OpenAI host instead of an Azure endpoint. In Goose, Azure requires AZURE_OPENAI_ENDPOINT plus deployment routing, so Azure-configured users will fail to generate summaries even with a valid key. This branch should build the Azure base URL (or reject unsupported Azure mode) rather than using the default OpenAI endpoint.
Useful? React with 👍 / 👎.
2 participants
Token Usage & Cost Estimator
Implements the token usage and cost estimator enhancement proposed in Phase 1.
What this adds:
Files added:
enhancements/token_cost_estimator/cost_estimator.pyenhancements/token_cost_estimator/pricing.jsonenhancements/token_cost_estimator/requirements.txtenhancements/token_cost_estimator/README.mdenhancements/token_cost_estimator/.gitignore