Skip to content

AI Request Logging only captures providers that use the SDK HTTP transporter; sidecar/custom-transport providers are invisible #732

@henryperkins

Description

@henryperkins

Summary

The AI Request Logging experiment captures requests by decorating the SDK's HTTP transporter. Any provider that performs its own HTTP (rather than routing generation through AiClient's transporter) is therefore never logged — even though it's a first-class provider reachable via wp_ai_client_prompt() and shown in the Connectors UI. The Request Log silently under-reports, which is surprising given it presents as a log of every AI request.

Root cause

Logging_Integration::wrap_transporter() installs the tap by swapping in a decorator:

// includes/Logging/Logging_Integration.php
$registry = AiClient::defaultRegistry();
$registry->setHttpTransporter( new Logging_Http_Transporter( $registry->getHttpTransporter(), $log_manager ) );

Logging then happens entirely inside Logging_Http_Transporter::send(). A request is logged iff it flows through HttpTransporter::send(). Provider attribution compounds this — Log_Data_Extractor::detect_provider() derives the provider from the request host, so a provider talking to 127.0.0.1 couldn't be attributed even if its request were seen.

Reproduction

  1. Enable the AI Request Logging experiment.
  2. Use a provider that doesn't route generation through the SDK transporter (e.g. one that proxies to a localhost sidecar or otherwise calls wp_remote_request() itself).
  3. Run a generation via wp_ai_client_prompt(). It succeeds, but no row appears in Tools → AI Request Log. The same site logs OpenAI/Anthropic generations correctly.

Impact

  • Any provider with a custom transport is omitted from the log — observability, cost/token accounting, and debugging all under-count.
  • Concrete case: a third-party codex provider that brokers requests through a localhost sidecar for ChatGPT-managed auth. It registers normally and serves generations, but is entirely absent from the Request Log. We've worked around it provider-side, but that means every such provider must reimplement logging and couple to AI_Request_Log_Manager internals — which is exactly what this issue asks to avoid.

Proposed fix

Add a provider-agnostic fallback alongside the transporter decorator: have the experiment also listen to the core generation events, which fire for every provider regardless of transport — wp_ai_client_before_generate_result / wp_ai_client_after_generate_result (bridged from the SDK's BeforeGenerateResultEvent / AfterGenerateResultEvent, with the dispatcher registered in wp-settings.php).

The after-event exposes everything a row needs:

  • getModel()->providerMetadata()->getId() — provider
  • getModel()->metadata()->getId() — model
  • getResult()->getTokenUsage() — tokens
  • duration = before → after

To avoid double-logging transporter-based providers (which fire the event and pass through the decorator), the listener should only write when the transporter did not already log the current generation — e.g. a flag reset in the before-event and set by Logging_Http_Transporter::send().

Known limitation: the SDK's after-event fires on success only (there is no error event), so this fallback captures successful non-transporter generations but not failed ones. That still closes the success case for all providers; failed custom-transport generations remain a separate gap that would need an SDK-level error event.

Alternatives considered

  • Provider-side logging (our current stopgap): works, but pushes coupling to AI_Request_Log_Manager into every provider and is easy to get wrong.
  • Hooking the WP HTTP API (pre_http_request): too broad, noisy, and can't reliably attribute the provider.

Prior art

#680 fixed a different "the logger misses data" case (thinking-token undercount) — same spirit of completeness.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions