Skip to content

feat(model-gateway): X-RLLM-* headers + inbound bearer auth#552

Open
jeffreysijuntan wants to merge 1 commit into
pr-1a-storefrom
pr-1b-metadata
Open

feat(model-gateway): X-RLLM-* headers + inbound bearer auth#552
jeffreysijuntan wants to merge 1 commit into
pr-1a-storefrom
pr-1b-metadata

Conversation

@jeffreysijuntan

Copy link
Copy Markdown
Contributor

Summary

Stacked on #551. Header-stamped session metadata becomes the canonical identity carrier; body fields and the legacy /sessions/{sid}/v1/... URL form are kept as field-by-field fallbacks. rLLM headers are stripped before forwarding upstream so they never leak to providers.

  • New metadata.py: RllmMetadata dataclass + extract_metadata() with precedence header > body > path, merged field-by-field (so a request can carry session_id in the URL and run_id in a header and both contribute).
  • Middleware attaches RllmMetadata to ASGI scope and gains an optional inbound-auth gate: when a bearer token is configured, every request must present Authorization: Bearer <token> (HMAC-compared) or get 401. Used by remote-sandbox eval where the gateway is exposed via a public tunnel.
  • TraceRecord gains the rLLM identity fields (run_id, harness, step_id, parent_span_id, span_type); data_process.build_trace_record populates them from RllmMetadata when present.
  • models.py carries the future-facing config fields (UpstreamRoute, routes, run_id, run_metadata, inbound_auth_token) so the upstream-proxy PR can wire them up without further model churn.

Stack

  • PR-1A — schema v2
  • [PR-1B] this — header metadata + inbound auth
  • PR-1C — upstream-proxy mode (depends on this)

Test plan

  • python -m pytest tests/unit/ — 242 passed
  • 15 new tests in test_metadata.py (header/body/path precedence, field-level merge, scope-shape decoding)
  • 7 new tests in test_inbound_auth.py (token absent → open, presented → 200, missing/wrong → 401, HMAC compare)

🤖 Generated with Claude Code

Header-stamped session metadata becomes the canonical identity carrier
for gateway requests. Body fields (`metadata.rllm.*`) and the legacy
`/sessions/{sid}/v1/...` URL form are kept as field-by-field
fallbacks; rLLM headers are stripped before forwarding upstream so
they never leak to providers.

Adds:
- `metadata.py` — `RllmMetadata` dataclass + `extract_metadata()` with
  precedence header > body > path, merged field-by-field (so a request
  can carry `session_id` in the URL and `run_id` in a header and both
  contribute).
- Middleware attaches `RllmMetadata` to ASGI scope and gains an
  optional inbound-auth gate: when a bearer token is configured, every
  request must present `Authorization: Bearer <token>` (HMAC-compared)
  or get 401. Used by remote-sandbox eval where the gateway is exposed
  via a public tunnel.
- TraceRecord gains the rLLM identity fields (`run_id`, `harness`,
  `step_id`, `parent_span_id`, `span_type`); `data_process.build_trace_record`
  populates them from `RllmMetadata` when present.
- `models.py` carries the future-facing config fields (`UpstreamRoute`,
  `routes`, `run_id`, `run_metadata`, `inbound_auth_token`) so the
  upstream-proxy PR can wire them up without further model churn.

Tests: 22 new (15 metadata precedence/extract, 7 inbound-auth).
Full unit suite: 242 passed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant