Skip to content

Commit 25f2b86

Browse files
Merge feat/agentos-auth-rbac-refresh: ingest cak_ auth + introspect path fix
2 parents 0bb2111 + ecf7fd2 commit 25f2b86

5 files changed

Lines changed: 184 additions & 16 deletions

File tree

CLAUDE.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ Resources stamped with `ownerGroup`/`ownerUser` are **hard-isolated**: a non-adm
108108
- On the **SDK** side: `AGENTOS_INGEST_URL` (e.g. `https://<host>/agentos/api/ingest/events`) + optional `AGENTOS_INGEST_TOKEN` (sent as `Authorization: Bearer …`). No Mongo creds.
109109
- On the **server** side: `MONGO_URL` + `MONGO_DATABASE` (this is the DB the collections above live in).
110110

111-
**Behaviour:** when `AGENTOS_INGEST_URL` is set, the SDK's default telemetry pipeline auto-attaches `AgentOSHttpSink` (gated on the `[agentos]` extra, which is now `httpx`-based). Each event carries a stable `event_id` so the server's writes are idempotent on retry. ⚠️ When the server's `AGENTOS_INGEST_TOKEN` is unset the ingest route is **open** (anonymous writes) — set it on any network-exposed deployment.
111+
**Behaviour:** when `AGENTOS_INGEST_URL` is set, the SDK's default telemetry pipeline auto-attaches `AgentOSHttpSink` (gated on the `[agentos]` extra, which is now `httpx`-based). Each event carries a stable `event_id` so the server's writes are idempotent on retry. **Ingest auth** (`ingest-auth.ts`): the SDK presents the **same `cak_` API key it uses everywhere** as the ingest `Bearer`, and the server validates it via `apiKeyStore.verify` (the same path the CAS introspection uses). A legacy static `AGENTOS_INGEST_TOKEN` string is still accepted for back-compat. ⚠️ Only when **no `cak_` is presented AND `AGENTOS_INGEST_TOKEN` is unset** does the route fall **open** (anonymous writes) — present a key (or set the token) on any network-exposed deployment. A presented `cak_` is always validated (invalid → 401, Mongo-down → 503), never waved through.
112112

113113
---
114114

@@ -122,7 +122,7 @@ Resources stamped with `ownerGroup`/`ownerUser` are **hard-isolated**: a non-adm
122122

123123
**Authorization — DB-backed roles.** Keycloak emits role *names* (`realm_access.roles`) + `groups`; AgentOS owns what each role *can do* via the `roles` collection (editable in Settings→Roles). `authenticate → resolvePermissions → authorize(perm)` gates every dashboard route. Permission catalog is code-defined (`auth/permissions.ts`).
124124

125-
**Three guards / trust boundaries** (`app.ts`): SERVICE `/agentos/api/ingest/*` (`requireIngestAuth`, fails open) + `/agentos/api/keys/*` (`requireIntrospectionAuth`, fails closed); DASHBOARD `/agentos/api/v1/*` (`authenticate`); OBS `/v1/*`. `cak_` API keys authenticate at the dashboard boundary too (→ service principal with `groups=[key.group]`).
125+
**Three guards / trust boundaries** (`app.ts`): SERVICE `/agentos/api/ingest/*` (`requireIngestAuth` — validates the presented `cak_` API key via `apiKeyStore`, legacy `AGENTOS_INGEST_TOKEN` as back-compat, open only when neither is present) + `/agentos/api/keys/*` (`requireIntrospectionAuth`, fails closed); DASHBOARD `/agentos/api/v1/*` (`authenticate`); OBS `/v1/*`. `cak_` API keys authenticate at the dashboard boundary too (→ service principal with `groups=[key.group]`).
126126

127127
**Groups = read-only from Keycloak Admin API** (Settings→Groups). If a user's token lacks the `groups` claim, the server backfills groups from the Admin API at login/refresh (`auth/keycloak-admin.ts:listUserGroups`).
128128

@@ -417,7 +417,7 @@ pnpm build && pnpm start # node dist/index.js
417417
| `COOKIE_SECURE` | derived from `NODE_ENV` | Force `true` / `false` explicitly |
418418
| `AGENTOS_SESSION_SECRET` | random per boot | HMAC secret for the signed BFF cookies (`agentos_session`/`agentos_refresh`). **Set to a stable value in prod** or every session is invalidated on restart |
419419
| `API_AUTH_USER` + `API_AUTH_PASS` | unset | **Legacy** — no longer gates the dashboard (SSO does, §2.6b). Now only used to build the Basic header for outbound loopback calls to the harness (`caAuthHeader`) |
420-
| `AGENTOS_INGEST_TOKEN` | unset | Bearer token guarding `POST /agentos/api/ingest/events` (the Python SDK's telemetry ingest). When unset the route is **open** (anonymous writes to registry/logs/sessions) — set it on any network-exposed pod. The SDK must send the same value as `AGENTOS_INGEST_TOKEN`. |
420+
| `AGENTOS_INGEST_TOKEN` | unset | **Legacy/back-compat** static Bearer for `POST /agentos/api/ingest/events`. Ingest now primarily validates the SDK's `cak_` **API key** (via `apiKeyStore`, the same key it presents to the CAS) — so the normal path needs no separate token: the SDK just sends its `cak_`. This static token is still accepted if presented verbatim. The route is **open** only when no `cak_` is presented *and* this is unset — set one or the other on any network-exposed pod. |
421421
| **Auth / RBAC** (§2.6b) || `KEYCLOAK_ISSUER_URL`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET` (+ optional `OIDC_AUDIENCE`/`OIDC_REDIRECT_URI`/`OIDC_POST_LOGOUT_URI`/`OIDC_ROLES_CLAIM`/`OIDC_GROUPS_CLAIM`); `AGENTOS_DEFAULT_ROLE`, `AGENTOS_BOOTSTRAP_ADMINS`, `AGENTOS_DEV_AUTH=1` (local only); `KEYCLOAK_ADMIN_CLIENT_ID`/`SECRET` for the Groups view + group backfill. Provision with `pnpm provision:keycloak`. |
422422
| **Git credentials** (§2.6c) | unset | `AGENTOS_CREDENTIALS_KEY` (base64 32B; **fail-closed** for credentials CRUD/resolve) + optional `AGENTOS_CREDENTIALS_KEY_OLD` for rotation. |
423423
| `AGENTOS_API_KEY_PEPPER` / `AGENTOS_INTROSPECTION_SECRET` | unset | HMAC pepper for `api_keys` hashing; shared secret guarding `/agentos/api/keys/introspect` (harness↔server) |
Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
// Unit tests for requireIngestAuth. api-key-store is mocked so the cak_
2+
// validation path is exercised without Mongo.
3+
4+
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
5+
6+
const h = vi.hoisted(() => ({ verify: vi.fn() }));
7+
vi.mock("./stores/api-key-store.js", () => ({
8+
apiKeyStore: { verify: h.verify },
9+
KEY_PREFIX: "cak_",
10+
}));
11+
12+
import { requireIngestAuth } from "./ingest-auth.js";
13+
14+
function ctx(authHeader?: string) {
15+
const req = {
16+
header: (n: string) => (n.toLowerCase() === "authorization" ? authHeader : undefined),
17+
} as unknown as import("express").Request;
18+
let statusCode = 0;
19+
let jsonBody: unknown;
20+
const res = {
21+
status(c: number) {
22+
statusCode = c;
23+
return this;
24+
},
25+
json(b: unknown) {
26+
jsonBody = b;
27+
return this;
28+
},
29+
} as unknown as import("express").Response;
30+
let passed = false;
31+
const next = () => {
32+
passed = true;
33+
};
34+
return {
35+
req,
36+
res,
37+
next,
38+
passed: () => passed,
39+
status: () => statusCode,
40+
body: () => jsonBody as { error?: { code?: string } },
41+
};
42+
}
43+
44+
const ORIGINAL = process.env["AGENTOS_INGEST_TOKEN"];
45+
46+
beforeEach(() => {
47+
h.verify.mockReset();
48+
delete process.env["AGENTOS_INGEST_TOKEN"];
49+
});
50+
51+
afterEach(() => {
52+
if (ORIGINAL === undefined) delete process.env["AGENTOS_INGEST_TOKEN"];
53+
else process.env["AGENTOS_INGEST_TOKEN"] = ORIGINAL;
54+
});
55+
56+
describe("requireIngestAuth — cak_ API key", () => {
57+
it("accepts a valid cak_ key (verified via apiKeyStore)", async () => {
58+
h.verify.mockResolvedValue({ active: true, principal: "key_abc", roleIds: [], scopes: ["*"] });
59+
const c = ctx("Bearer cak_validkey");
60+
await requireIngestAuth(c.req, c.res, c.next);
61+
expect(c.passed()).toBe(true);
62+
expect(h.verify).toHaveBeenCalledWith("cak_validkey");
63+
});
64+
65+
it("rejects an inactive/unknown cak_ key with 401", async () => {
66+
h.verify.mockResolvedValue(null);
67+
const c = ctx("Bearer cak_revoked");
68+
await requireIngestAuth(c.req, c.res, c.next);
69+
expect(c.passed()).toBe(false);
70+
expect(c.status()).toBe(401);
71+
expect(c.body().error?.code).toBe("UNAUTHENTICATED");
72+
});
73+
74+
it("fails closed with 503 when key verification throws (Mongo down)", async () => {
75+
h.verify.mockRejectedValue(new Error("connection refused"));
76+
const c = ctx("Bearer cak_whatever");
77+
await requireIngestAuth(c.req, c.res, c.next);
78+
expect(c.passed()).toBe(false);
79+
expect(c.status()).toBe(503);
80+
expect(c.body().error?.code).toBe("KEY_VERIFICATION_UNAVAILABLE");
81+
});
82+
83+
it("never consults the legacy token for a cak_ key", async () => {
84+
process.env["AGENTOS_INGEST_TOKEN"] = "cak_validkey"; // even if it matches verbatim
85+
h.verify.mockResolvedValue(null);
86+
const c = ctx("Bearer cak_validkey");
87+
await requireIngestAuth(c.req, c.res, c.next);
88+
expect(c.status()).toBe(401); // a cak_ must be a real key, not string-equal to the legacy token
89+
});
90+
});
91+
92+
describe("requireIngestAuth — legacy static token (back-compat)", () => {
93+
it("accepts a matching AGENTOS_INGEST_TOKEN", async () => {
94+
process.env["AGENTOS_INGEST_TOKEN"] = "s3cr3t";
95+
const c = ctx("Bearer s3cr3t");
96+
await requireIngestAuth(c.req, c.res, c.next);
97+
expect(c.passed()).toBe(true);
98+
expect(h.verify).not.toHaveBeenCalled();
99+
});
100+
101+
it("rejects a wrong token with 401", async () => {
102+
process.env["AGENTOS_INGEST_TOKEN"] = "s3cr3t";
103+
const c = ctx("Bearer nope");
104+
await requireIngestAuth(c.req, c.res, c.next);
105+
expect(c.status()).toBe(401);
106+
});
107+
108+
it("rejects when token is configured but none is presented", async () => {
109+
process.env["AGENTOS_INGEST_TOKEN"] = "s3cr3t";
110+
const c = ctx(undefined);
111+
await requireIngestAuth(c.req, c.res, c.next);
112+
expect(c.status()).toBe(401);
113+
});
114+
});
115+
116+
describe("requireIngestAuth — open mode", () => {
117+
it("passes when no token is configured and none is presented (network policy)", async () => {
118+
const c = ctx(undefined);
119+
await requireIngestAuth(c.req, c.res, c.next);
120+
expect(c.passed()).toBe(true);
121+
});
122+
123+
it("passes a non-cak bearer when no token is configured", async () => {
124+
const c = ctx("Bearer some-random-thing");
125+
await requireIngestAuth(c.req, c.res, c.next);
126+
expect(c.passed()).toBe(true);
127+
});
128+
});
Lines changed: 48 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,60 @@
11
// Machine-to-machine auth for the telemetry ingest endpoint.
22
//
33
// The dashboard's cookie/Basic `requireAuth` is for browsers; the Python SDK
4-
// posts events headless, so the ingest route gets its own bearer-token guard
5-
// keyed on AGENTOS_INGEST_TOKEN. When the env is unset the route is OPEN
6-
// (relies on network policy) — same philosophy as API_AUTH_* in auth.ts.
4+
// posts events headless. The SDK uses ONE credential everywhere — the `cak_`
5+
// API key it already presents to the ComputerAgent server — so ingest verifies
6+
// that same key (via `apiKeyStore`, the same path the CAS introspection uses)
7+
// rather than a separate shared secret.
8+
//
9+
// Auth resolution order:
10+
// 1. Bearer `cak_…` → validate against `api_keys` (active/not
11+
// revoked/not expired). Invalid → 401.
12+
// 2. Bearer <AGENTOS_INGEST_TOKEN> → legacy shared-secret, kept for
13+
// back-compat with older deployments.
14+
// 3. Neither presented AND no AGENTOS_INGEST_TOKEN configured → OPEN
15+
// (relies on network policy — same philosophy
16+
// as API_AUTH_* in auth.ts). Once a `cak_` is
17+
// presented it is always validated, never
18+
// waved through.
719

820
import type { RequestHandler } from "express";
921
import { timingSafeEqual } from "node:crypto";
22+
import { apiKeyStore, KEY_PREFIX } from "./stores/api-key-store.js";
1023

11-
export const requireIngestAuth: RequestHandler = (req, res, next) => {
12-
const expected = process.env["AGENTOS_INGEST_TOKEN"];
13-
if (!expected) return next(); // open — network policy only
24+
function unauthenticated(res: Parameters<RequestHandler>[1]): void {
25+
res.status(401).json({ error: { code: "UNAUTHENTICATED" } });
26+
}
1427

28+
export const requireIngestAuth: RequestHandler = async (req, res, next) => {
1529
const header = req.header("authorization") ?? "";
1630
const prefix = "Bearer ";
17-
if (header.startsWith(prefix)) {
18-
const got = Buffer.from(header.slice(prefix.length), "utf8");
19-
const want = Buffer.from(expected, "utf8");
20-
if (got.length === want.length && timingSafeEqual(got, want)) return next();
31+
const presented = header.startsWith(prefix) ? header.slice(prefix.length) : "";
32+
33+
// (1) API key — the same `cak_` key the SDK uses everywhere.
34+
if (presented.startsWith(KEY_PREFIX)) {
35+
try {
36+
const result = await apiKeyStore.verify(presented);
37+
if (result) return next();
38+
return unauthenticated(res); // recognized shape, but inactive/revoked/unknown
39+
} catch (err) {
40+
// Validation infra (Mongo) is down — fail closed, but distinguish from a
41+
// bad key so the caller can retry rather than treating it as a 401.
42+
console.warn("[agentos-server] ingest key verification failed:", (err as Error).message);
43+
return res.status(503).json({ error: { code: "KEY_VERIFICATION_UNAVAILABLE" } });
44+
}
2145
}
22-
res.status(401).json({ error: { code: "UNAUTHENTICATED" } });
46+
47+
// (2) Back-compat: legacy static shared ingest token.
48+
const expected = process.env["AGENTOS_INGEST_TOKEN"];
49+
if (expected) {
50+
if (presented) {
51+
const got = Buffer.from(presented, "utf8");
52+
const want = Buffer.from(expected, "utf8");
53+
if (got.length === want.length && timingSafeEqual(got, want)) return next();
54+
}
55+
return unauthenticated(res);
56+
}
57+
58+
// (3) Nothing presented and nothing configured → open (network policy only).
59+
return next();
2360
};

packages/agentos-server/src/routes/keys-introspect.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,9 @@ import { resolveEffectivePermissions } from "../auth/authorize.js";
99

1010
export const keysIntrospectRouter: IRouter = Router();
1111

12-
keysIntrospectRouter.post("/keys/introspect", async (req, res, next) => {
12+
// Mounted at `/agentos/api/keys`, so this is `/agentos/api/keys/introspect`
13+
// (matches the CAS verifier URL + every doc/comment referencing the endpoint).
14+
keysIntrospectRouter.post("/introspect", async (req, res, next) => {
1315
try {
1416
const key = (req.body as { key?: unknown } | undefined)?.key;
1517
if (typeof key !== "string" || !key) {

packages/agentos-server/src/stores/telemetry-projection.test.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,8 @@ describe("projectEvent — full run", () => {
130130
// agent_registry — library source (harness_mode), prefix-stripped model.
131131
const reg = docsOf("agent_registry");
132132
expect(reg).toHaveLength(1);
133-
expect(reg[0]._id).toBe("bot-A");
133+
// Registry is ObjectId-keyed; `name` is the unique business key.
134+
expect(reg[0].name).toBe("bot-A");
134135
expect(reg[0].harness).toBe("claude-agent-sdk");
135136
expect(reg[0].source.type).toBe("library");
136137
expect(reg[0].model).toBe("claude-x");

0 commit comments

Comments
 (0)