Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions docs/SECURITY_INTEGRATION_TESTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,67 @@ All 30 tests pass successfully, covering:
- ✅ End-to-end vault lifecycle flows
- ✅ Audit logging and compliance features

---

## RBAC Role-Matrix Tests (Issue #623)

A comprehensive role-matrix block added to the same file systematically
exercises every `/api/admin/*` endpoint across all three roles and the
unauthenticated case.

### Endpoint / Role Matrix

| Endpoint | Method | ADMIN | USER | VERIFIER | Unauth |
|---|---|---|---|---|---|
| `/api/admin/users` | GET | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/users/:id/role` | PATCH | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/users/:id/status` | PATCH | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/users/:id` | DELETE | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/users/:id/restore` | POST | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/audit-logs` | GET | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/audit-logs/:id` | GET | ✅ 404* | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/overrides/vaults/:id/cancel` | POST | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/users/:userId/revoke-sessions` | POST | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/verifiers` | GET | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/verifiers/:userId` | GET | ✅ 404* | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/verifiers` | POST | ✅ 201 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/verifiers/:userId` | PATCH | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/verifiers/:userId` | DELETE | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/verifiers/:userId/approve` | POST | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/admin/verifiers/:userId/suspend` | POST | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |
| `/api/verifications` | POST | ✅ 201 | ❌ 403 | ✅ 201 | ❌ 401 |
| `/api/verifications` | GET | ✅ 200 | ❌ 403 | ❌ 403 | ❌ 401 |

\* 404 is an expected business-logic response from the admin handler — not an RBAC error.

### Security Invariants Tested

- **Role from JWT only** — 5 header-spoofing combinations (x-user-role, x-requested-role,
role, x-auth-role, multiple combined). Both "USER token + spoof header" and
"no token + spoof header" are verified to never grant elevated access.
- **Auth before authz** — missing token, malformed token, wrong-secret token, and expired
token all return 401 (never 403).
- **Error envelope consistency** — 401 and 403 responses both carry `{ error: string }`.
403 responses optionally include a `message` field naming the required role.
- **Path-param edge cases** — non-existent vault/log/verifier IDs return 404 under an
admin token, confirming RBAC passed and only business logic rejected the request.

### Additional Test Groups (original suite)

See table in [Test Coverage](#test-coverage) above.

### Test Count Summary

| Group | Tests |
|---|---|
| Original suite | 30 |
| RBAC Role-Matrix (Issue #623) | 92 |
| **Total** | **122** |

2 tests in the original suite have pre-existing failures unrelated to RBAC (they test a
specific `res.body.error.code` shape that the minimal test-app does not produce). All 92
new role-matrix tests pass.

## Security Considerations

### No Secrets in Repository
Expand Down
94 changes: 94 additions & 0 deletions docs/export.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,97 @@ CSV downloads are emitted as UTF-8 with a leading BOM so spreadsheet tools such
| File storage | `Buffer` in memory | S3 / GCS pre-signed URLs |
| Download secret | Env var `DOWNLOAD_SECRET` | AWS Secrets Manager / Vault |
| Data source | Shared in-memory array | Parameterised DB queries per user |

---

## Dead-Letter Queue (DLQ)

When an export job exhausts all retry attempts it is moved to an in-memory DLQ. The DLQ is queryable and drainable at runtime via service methods — no API surface change is required.

### DLQ Entry structure (`DlqEntry`)

```ts
interface DlqEntry {
jobId: string // original ExportJob id
jobType: string // "scope:format", e.g. "vaults:csv"
failureReason: FailureReason
errorMessage: string
attemptCount: number
failedAt: string // ISO-8601 UTC
sanitisedContext: {
userToken: string // first 8 chars of SHA-256(userId) — no raw PII
targetUserToken?: string // first 8 chars of SHA-256(targetUserId) if set
scope: ExportScope
format: ExportFormat
}
}
```

`FailureReason` is one of `serialization_error | data_fetch_error | unknown_error` and is
classified automatically from the caught error message.

### DLQ capacity

The DLQ is capped at `maxDlqSize` entries (default **100**). When the cap is reached the
oldest entry is evicted before the new one is inserted. Configure at startup:

```ts
import { configureDlq } from './services/exportQueue.js'
configureDlq({ maxDlqSize: 200 })
```

### Query API

| Method | Description |
|---|---|
| `getDlqEntries()` | Snapshot of all entries, newest-first. Mutations to the returned array do not affect the store. |
| `getDlqEntry(jobId)` | Single entry or `undefined`. |
| `getDlqDepth()` | Current entry count. |

### Drain API

| Method | Returns | Description |
|---|---|---|
| `requeueDlqEntry(jobId)` | `Promise<boolean>` | Removes from DLQ and re-creates the job as `pending` with reset attempts. Returns `false` if `jobId` not found. |
| `discardDlqEntry(jobId)` | `boolean` | Permanently removes entry. Returns `false` if not found. |
| `clearDlq()` | `number` | Removes all entries; returns count of removed entries. |

### Optional metrics hook

Register a callback at startup to receive a `DlqMetricsEvent` on every DLQ mutation:

```ts
import { configureDlq, type DlqMetricsEvent } from './services/exportQueue.js'

configureDlq({
metricsHook: (event: DlqMetricsEvent) => {
// event.event — 'dlq.entry_added' | 'dlq.entry_requeued' | 'dlq.entry_discarded' | 'dlq.cleared'
// event.jobId — affected job id (empty string for 'dlq.cleared')
// event.dlqDepth — depth after the mutation
// event.timestamp — ISO-8601 UTC
myMetricsClient.gauge('export.dlq.depth', event.dlqDepth)
}
})
```

A throwing hook is caught and logged at `warn` level — it never interrupts normal queue
operation.

### Structured log lines emitted by the DLQ

| Event | Level | Key fields |
|---|---|---|
| Entry added | `warn` | `jobId`, `failureReason`, `errorMessage`, `attemptCount`, `dlqDepth` |
| Entry requeued | `info` | `jobId`, `dlqDepth` |
| Entry discarded | `info` | `jobId`, `dlqDepth` |
| DLQ cleared | `info` | `count`, `dlqDepth` |

All log lines are structured JSON and contain **no raw `userId` or `targetUserId`**.

### PII contract

- `userId` and `targetUserId` are replaced by a deterministic opaque token (first 8 hex chars
of SHA-256) before storage in `DlqEntry.sanitisedContext`.
- Raw Stellar account addresses, email addresses, and any field classified as PII in
`PRIVACY.md` are never written to a `DlqEntry`.
- The metrics hook receives only the sanitised event — no PII is emitted via the hook.
Loading
Loading