Problem / motivation
I'd like to understand what tools are being used, both built-in and also other tools such as MCP servers (maybe skills too??), at a session level and also across projects. I've been working on incorporating other tools into my workflows (currently focused on MCP servers such as agent memory, code understanding, documention tools, etc) and tweaking my CLAUDE.md files to guide the agent using "routing" rules for tools. It would be helpful to be able to see tool usage within a session (ie is it being used, how much) but also over time across sessions as I tweak my routing rules. I'd also like to understand which tools (especially MCP servers) are never getting used such that I can remove them completely to reduce context. It would also be useful if tool usage would be quantified to token/cost -- if I have two tools that do roughly the same thing (such as the built-in Explore and IntelliJ as an MCP server) which is the most cost effective.
A "stretch goal" ;) might be to even add this to some of the LLM analysis under Insights -- how often is a tool used that doesn't provide the agent with what it needs so it reverts to another tool or it has to make multiple additional tool calls to the same too to refine to get to the answer
Proposed solution
Add additional analytics around tool usage.
Alternatives considered
Using the Agentsview API and a prompt to extract the detail out
Problem / motivation
I'd like to understand what tools are being used, both built-in and also other tools such as MCP servers (maybe skills too??), at a session level and also across projects. I've been working on incorporating other tools into my workflows (currently focused on MCP servers such as agent memory, code understanding, documention tools, etc) and tweaking my CLAUDE.md files to guide the agent using "routing" rules for tools. It would be helpful to be able to see tool usage within a session (ie is it being used, how much) but also over time across sessions as I tweak my routing rules. I'd also like to understand which tools (especially MCP servers) are never getting used such that I can remove them completely to reduce context. It would also be useful if tool usage would be quantified to token/cost -- if I have two tools that do roughly the same thing (such as the built-in Explore and IntelliJ as an MCP server) which is the most cost effective.
A "stretch goal" ;) might be to even add this to some of the LLM analysis under Insights -- how often is a tool used that doesn't provide the agent with what it needs so it reverts to another tool or it has to make multiple additional tool calls to the same too to refine to get to the answer
Proposed solution
Add additional analytics around tool usage.
Alternatives considered
Using the Agentsview API and a prompt to extract the detail out