Add AI binding support for pi

.github/workflows/preview.yml

@@ -106,13 +106,20 @@ jobs:
           CF_ACCESS_ISS: ${{ secrets.CF_ACCESS_ISS }}
           # AI Gateway, so a preview's chats use server-managed keys rather than asking each user
           # for their own. Optional as a group: with CF_AI_GATEWAY unset the preview is BYOK, but
-          # once it is set the account id and the Run + Read token are required.
+          # once it is set the account id is required.
           CF_AI_GATEWAY: ${{ secrets.CF_AI_GATEWAY }}
           # This is delibaretely set to CF_OS_AI_GATEWAY_ACCOUNT_ID (gets uploaded as CF_AI_GATEWAY_ACCOUNT_ID)
           CF_AI_GATEWAY_ACCOUNT_ID: ${{ secrets.CF_OS_AI_GATEWAY_ACCOUNT_ID }}
-          CF_AI_GATEWAY_API_TOKEN: ${{ secrets.CF_AI_GATEWAY_API_TOKEN }}
           CF_AI_GATEWAY_PROVIDERS: ${{ secrets.CF_AI_GATEWAY_PROVIDERS }}
-          CF_AI_GATEWAY_WAI_DIRECT: ${{ secrets.CF_AI_GATEWAY_WAI_DIRECT }}
+          # The gateway above lives in the account these previews deploy to, so the backend's
+          # WORKERS_AI binding reaches it: inference and cost-log reads are pre-authenticated
+          # in-account, which is why no CF_AI_GATEWAY_API_TOKEN is passed at all.
+          #
+          # Two configurations cannot use the binding and so are unreachable from here until a
+          # `CF_AI_GATEWAY_API_TOKEN: ${{ secrets.CF_AI_GATEWAY_API_TOKEN }}` line is added back:
+          # a gateway in a *different* account (which sets this secret to `false`), and the google
+          # provider, whose SDK cannot take the binding's fetch. Both fail the deploy.
+          CF_AI_GATEWAY_USE_BINDING: ${{ secrets.CF_AI_GATEWAY_USE_BINDING || 'true' }}
           # Together these are the preview name, which is the first label of its hostname: a preview
           # reads as `pr123-my-branch-router.<subdomain>.workers.dev`. The number is what keeps two
           # branches that slugify alike from sharing one instance, and is how the nightly sweep

docs/ai-gateway-billing.md

@@ -13,10 +13,9 @@ turn, the overseer calls `checkUsageAndBalance`:
 - **Connected, balance ≥ `$2`** → allowed, routed through the user's own account so usage bills
   their Cloudflare credits — even while free-tier allowance remains. The platform is never charged
   for funded users, and their daily free-tier counter is left untouched.
-- **Otherwise, within the free tier** → allowed, served via the platform's configured AI Gateway.
-  Workers AI uses the same Gateway ID unless `CF_AI_GATEWAY_WAI_DIRECT=true` sends it straight to
-  the Workers AI REST endpoint or `CF_AI_GATEWAY_WAI` selects another Gateway. This includes
-  connected users whose balance is below `$2` (incl. $0).
+- **Otherwise, within the free tier** → allowed, served via the platform's configured AI Gateway
+  (all providers, Workers AI included). This includes connected users whose balance is below `$2`
+  (incl. $0).
 - **Free tier exhausted, no Cloudflare account connected** → blocked, with a prompt to connect.
 - **Free tier exhausted, connected but balance below `$2`** → blocked, with a prompt to add credits.
 
@@ -54,19 +53,24 @@ CLOUDFLARE_OAUTH_CLIENT_SECRET=...
 CF_AI_GATEWAY=your-gateway
 CF_AI_GATEWAY_PROVIDERS=anthropic,openai,google
 
-# Required whenever CF_AI_GATEWAY is set (all inference goes over HTTPS with tokens):
+# Required whenever CF_AI_GATEWAY is set:
 CF_AI_GATEWAY_ACCOUNT_ID=...
+# Required unless the WORKERS_AI binding carries gateway traffic; always required for the
+# google provider:
 CF_AI_GATEWAY_API_TOKEN=...
-
-# To send Workers AI straight to its REST endpoint (no gateway, no cost logs):
-CF_AI_GATEWAY_WAI_DIRECT=true
 ```
 
-Gateway mode always requires `CF_AI_GATEWAY_ACCOUNT_ID` and an API token with AI Gateway Run and
-Read permissions; Read access lets Gadgets retrieve each log's cost for user-visible accounting.
-Workers AI uses `CF_AI_GATEWAY` as its Gateway ID by default; set `CF_AI_GATEWAY_WAI` to select
-another Gateway, or `CF_AI_GATEWAY_WAI_DIRECT=true` to call the Workers AI REST endpoint directly
-(same credentials, no gateway cost logs).
+Gateway mode always requires `CF_AI_GATEWAY_ACCOUNT_ID` plus a transport: the `WORKERS_AI`
+binding when present (binding requests are pre-authenticated, and cost-log reads work through
+the binding too), or otherwise an API token with AI Gateway Run and Read permissions — Read
+access lets Gadgets retrieve each log's cost for user-visible accounting. The binding transport
+only works when the Gateway lives in the Worker's own account, which the Worker can't verify at
+runtime — a deployment whose Gateway is in a different account must set
+`CF_AI_GATEWAY_USE_BINDING=false` to opt out and use the token transport. That is a flag rather
+than an unbinding because `WORKERS_AI` also backs the webFetch tool's document-to-Markdown
+conversion (and is hardcoded for every released backend), so removing it would break that instead
+of just moving gateway traffic. The token stays required for the `google` provider even when the
+binding transport applies. Every provider, Workers AI included, routes through the same Gateway.
 
 The Cloudflare dashboard OAuth endpoints and scopes are **hardcoded** in the Cloudflare gatekeeper
 (`packages/gatekeeper-cloudflare/src/oauth.ts`):

docs/public-server.md

@@ -40,25 +40,35 @@ CLOUDFLARE_OAUTH_CLIENT_SECRET=...
 CF_AI_GATEWAY=your-gateway
 CF_AI_GATEWAY_PROVIDERS=anthropic,openai,google
 
-# Required whenever CF_AI_GATEWAY is set (all inference goes over HTTPS with tokens):
+# Required whenever CF_AI_GATEWAY is set:
 CF_AI_GATEWAY_ACCOUNT_ID=...
+# Required unless the WORKERS_AI binding carries gateway traffic (see below); always required
+# for the google provider:
 CF_AI_GATEWAY_API_TOKEN=...
-
-# To send Workers AI straight to its REST endpoint (no gateway, no cost logs):
-CF_AI_GATEWAY_WAI_DIRECT=true
 ```
 
-Gateway mode always requires `CF_AI_GATEWAY_ACCOUNT_ID` and `CF_AI_GATEWAY_API_TOKEN`; the token
-needs AI Gateway Run and Read permissions so Gadgets can execute models and report their costs
-(the Gateway may live in the Worker's own account or a different one). Workers AI defaults to the
-same Gateway ID; set `CF_AI_GATEWAY_WAI` to route it through a different Gateway in the same
-account, or `CF_AI_GATEWAY_WAI_DIRECT=true` to bypass gateways and call the Workers AI REST
-endpoint directly (using the same account/token pair; such requests produce no cost logs).
+Gateway mode always requires `CF_AI_GATEWAY_ACCOUNT_ID`, plus a transport for gateway requests.
+When the `WORKERS_AI` binding is present, the binding is that transport by default: its requests
+are pre-authenticated in-account, so inference and cost-log reads need no API token. This is only
+valid when the Gateway lives in the Worker's **own** account — binding requests can't reach
+another account's Gateway, and the Worker cannot verify where the Gateway lives at runtime — so
+deployments whose Gateway is in a different account must set `CF_AI_GATEWAY_USE_BINDING=false` to
+opt out and route over HTTPS instead. Keep `WORKERS_AI` bound when you do: it is also what the
+webFetch tool's document-to-Markdown conversion runs on, so unbinding it opts out of far more than
+the gateway transport. Without the binding transport, set
+`CF_AI_GATEWAY_API_TOKEN` — a token with AI Gateway Run and Read permissions so Gadgets can
+execute models and report their costs (over HTTPS the Gateway may live in the Worker's own
+account or a different one). The token stays required for the `google` provider regardless of the
+binding (the model SDK adapter refuses the binding's fetch — note the platform config above enables
+it, so the platform server itself still needs the token). Every provider, Workers AI included,
+routes through the same Gateway.
 
 When using `CF_AI_GATEWAY*` in local development, start the server with
-`pnpm run dev-server -- --use-workers-ai-binding` so the webFetch tool's document-to-Markdown
-conversion still has a `WORKERS_AI` binding. (Inference itself no longer uses the binding; it goes
-over HTTPS with the tokens above.)
+`pnpm run dev-server -- --use-workers-ai-binding` so the server has a `WORKERS_AI` binding for
+the webFetch tool's document-to-Markdown conversion and for the gateway transport above (without
+it, gateway traffic falls back to HTTPS with `CF_AI_GATEWAY_API_TOKEN`). If your dev Gateway
+lives in a different account than the binding, also set `CF_AI_GATEWAY_USE_BINDING=false` — keep
+`--use-workers-ai-binding` on, since the Markdown conversion still needs the binding.
 
 Each gatekeeper's OAuth app must be registered with that gatekeeper's redirect URI (replace the host
 with `PUBLIC_BASE_URL`):

packages/workshop-backend/__tests__/ai-gateway.test.ts

@@ -1,5 +1,6 @@
 import { afterEach, describe, expect, it, vi } from "vitest";
 import {
+  AiGatewayConfig,
   AiGatewayLogRetryableError,
   getAiGatewayLogCost,
 } from "../src/ai-gateway.js";
@@ -13,6 +14,114 @@ function env(overrides: Partial<Cloudflare.Env> = {}): Cloudflare.Env {
   } as Cloudflare.Env;
 }
 
+describe("AiGatewayConfig transport selection", () => {
+  const binding = { gateway: () => ({}) } as unknown as Ai;
+  // google needs the HTTPS+token transport, so token-less configs must not enable it.
+  const bindingOnly = env({
+    CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+    CF_AI_GATEWAY_PROVIDERS: "anthropic,openai,cloudflare",
+    WORKERS_AI: binding,
+  });
+
+  it("uses the binding for every provider except google", () => {
+    const config = new AiGatewayConfig(bindingOnly);
+    expect(config.apiToken).toBeUndefined();
+    expect(config.bindingFor("anthropic")).toBe(binding);
+    expect(config.bindingFor("openai")).toBe(binding);
+    expect(config.bindingFor("cloudflare")).toBe(binding);
+    expect(config.bindingFor("google")).toBeUndefined();
+  });
+
+  it("falls back to HTTPS with the token when the binding is absent", () => {
+    const config = new AiGatewayConfig(env({
+      CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+      CF_AI_GATEWAY_API_TOKEN: "gateway-token",
+      WORKERS_AI: undefined,
+    }));
+    expect(config.apiToken).toBe("gateway-token");
+    expect(config.bindingFor("anthropic")).toBeUndefined();
+  });
+
+  it("ignores the binding when CF_AI_GATEWAY_USE_BINDING=false opts out", () => {
+    // The cross-account shape (e.g. the internal production Workshop): WORKERS_AI is injected
+    // for webFetch, but the gateway lives in a different account, so the deployment opts out
+    // and gateway traffic rides HTTPS with the token.
+    const config = new AiGatewayConfig(env({
+      CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+      CF_AI_GATEWAY_API_TOKEN: "gateway-token",
+      CF_AI_GATEWAY_USE_BINDING: "false",
+      WORKERS_AI: binding,
+    }));
+    expect(config.binding).toBeUndefined();
+    expect(config.apiToken).toBe("gateway-token");
+    expect(config.bindingFor("anthropic")).toBeUndefined();
+    expect(config.bindingFor("openai")).toBeUndefined();
+  });
+
+  it("opts out on a padded, mixed-case CF_AI_GATEWAY_USE_BINDING", () => {
+    const config = new AiGatewayConfig(env({
+      CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+      CF_AI_GATEWAY_API_TOKEN: "gateway-token",
+      CF_AI_GATEWAY_USE_BINDING: " False ",
+      WORKERS_AI: binding,
+    }));
+    expect(config.binding).toBeUndefined();
+    expect(config.bindingFor("anthropic")).toBeUndefined();
+  });
+
+  it("still requires a transport when the opt-out leaves no token", () => {
+    expect(() => new AiGatewayConfig({
+      ...bindingOnly,
+      CF_AI_GATEWAY_USE_BINDING: "false",
+    })).toThrow("AI Gateway mode needs a transport");
+  });
+
+  it("rejects an explicit CF_AI_GATEWAY_USE_BINDING=true without the WORKERS_AI binding", () => {
+    expect(() => new AiGatewayConfig(env({
+      CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+      CF_AI_GATEWAY_API_TOKEN: "gateway-token",
+      CF_AI_GATEWAY_USE_BINDING: "true",
+      WORKERS_AI: undefined,
+    }))).toThrow("CF_AI_GATEWAY_USE_BINDING requires the WORKERS_AI binding");
+  });
+
+  it("requires the account id", () => {
+    expect(() => new AiGatewayConfig(env({ CF_AI_GATEWAY_ACCOUNT_ID: undefined })))
+        .toThrow("CF_AI_GATEWAY_ACCOUNT_ID is required when CF_AI_GATEWAY is set.");
+  });
+
+  it("requires a transport", () => {
+    expect(() => new AiGatewayConfig(env({
+      CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+      WORKERS_AI: undefined,
+    }))).toThrow("AI Gateway mode needs a transport");
+  });
+
+  it("requires the token when google is enabled", () => {
+    expect(() => new AiGatewayConfig(env({
+      CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+      WORKERS_AI: binding,
+    }))).toThrow("enabling the google provider requires CF_AI_GATEWAY_API_TOKEN");
+  });
+
+  it("resolves the same-account gateway for binding-based callers (webFetch)", () => {
+    expect(new AiGatewayConfig(bindingOnly).sameAccountGateway).toBe("platform-gateway");
+    expect(new AiGatewayConfig(env({
+      CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+      CF_AI_GATEWAY_API_TOKEN: "gateway-token",
+      CF_AI_GATEWAY_USE_BINDING: "false",
+      WORKERS_AI: binding,
+    })).sameAccountGateway).toBeUndefined();
+    // It tracks the binding rather than the opt-out, so an HTTPS-only deployment that never had a
+    // binding to opt out of resolves no same-account gateway either.
+    expect(new AiGatewayConfig(env({
+      CF_AI_GATEWAY_ACCOUNT_ID: "account-id",
+      CF_AI_GATEWAY_API_TOKEN: "gateway-token",
+      WORKERS_AI: undefined,
+    })).sameAccountGateway).toBeUndefined();
+  });
+});
+
 describe("getAiGatewayLogCost", () => {
   afterEach(() => vi.unstubAllGlobals());