feat: generic webhook adapter for arbitrary detectors#108
Merged
Conversation
Adds a new endpoint at POST /v1/signals/webhook/{name} that accepts arbitrary
JSON from any detector or telemetry source and maps it to an AttackEvent via
operator-defined JSONPath expressions. No Rust code required for new integrations.
Config (correlation.yaml):
- webhook_adapters[] with per-adapter name, auth, field mappings, vector
normalization, and confidence scaling
- Auth modes: hmac (HMAC-SHA256, constant-time, env-loaded secret), bearer
(reuses session auth), or none
- root_path enables batched payloads (iterate an array; one event per match)
Backend (src/correlation/webhook.rs):
- WebhookAdapter, WebhookFieldMap, WebhookAuth structs (utoipa schemas)
- CompiledAdapter pre-parses JSONPath expressions per adapter
- map_payload extracts and normalizes fields into AttackEventInput
- verify_hmac_sha256 with constant-time compare via subtle crate
- 13 unit tests for mapping and HMAC
API handler (src/api/handlers.rs):
- ingest_webhook resolves adapter by name, verifies auth, parses body,
feeds each mapped event through the existing ban/unban pipeline
- Disabled or unknown adapters return 404; name validation prevents path
traversal via [a-z0-9-]{1,64} restriction
- 11 integration tests covering happy path, HMAC positive/negative, 404s,
malformed JSON, batching, missing required fields, vector mapping
Deps: adds serde_json_path 0.7 (RFC 9535 JSONPath) and subtle 2
Frontend: - New WebhookAdaptersEditor (full CRUD) wired into the Correlation Config tab - Supports all three auth modes (hmac/bearer/none), JSONPath field mapping, vector_map / confidence_scale / source_id_prefix / root_path - Client-side validator with 10 Vitest cases - Adds WebhookAdapter, WebhookAuth, WebhookFieldMap types to lib/api.ts Docs: - New ADR 020: rationale for generic webhook adapter (JSONPath, per-adapter auth, secret handling, endpoint shape, no-migration decision) - New docs/detectors/generic-webhook.md: end-to-end Radware walkthrough, JSONPath cheat sheet, troubleshooting, security notes - README: new adapter callout + feature-table entry + data-flow note - AGENTS.md: endpoint listed, module note, ADR count bumped to 20 - FEATURES.md: generic webhook adapter entry under Signal Ingestion - ROADMAP.md: generic webhook adapter shipped; transform-functions + commercial appliance recipes added to backlog; FastNetMon native adapter marked done - docs/adr/README.md: index entry for ADR 020
There was a problem hiding this comment.
Pull request overview
Adds a configuration-driven “generic webhook adapter” so operators can ingest arbitrary JSON detectors via POST /v1/signals/webhook/{name} without writing new Rust adapters, including JSONPath field mapping and per-adapter auth.
Changes:
- Backend: introduce
src/correlation/webhook.rs, extendCorrelationConfigwithwebhook_adapters, add webhook ingestion route/handler + OpenAPI types. - Frontend: add
WebhookAdaptersEditorCRUD UI and validator tests. - Docs/config/examples: ADR 020 + detector guide + config/API documentation updates.
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/integration.rs | Adds integration tests for the new webhook endpoint (happy path, batching, auth, error reporting). |
| tests/common/mod.rs | Updates E2E context to include webhook_adapters default. |
| src/correlation/webhook.rs | Implements adapter config structs, JSONPath compilation/mapping, name validation, and HMAC verification. |
| src/correlation/mod.rs | Exposes webhook module and re-exports key types/helpers. |
| src/correlation/config.rs | Adds webhook_adapters to config, validation, and redacted API view. |
| src/api/routes.rs | Registers POST /v1/signals/webhook/{name} route. |
| src/api/openapi.rs | Adds webhook handler + schemas to the OpenAPI spec. |
| src/api/handlers.rs | Implements ingest_webhook handler and response types. |
| frontend/lib/api.ts | Adds TS types for webhook adapters and extends CorrelationConfig. |
| frontend/components/dashboard/correlation/webhook-adapters.tsx | Adds full CRUD editor UI for webhook adapters. |
| frontend/components/dashboard/correlation/config-tab.tsx | Mounts the webhook adapters editor in the Correlation config tab. |
| frontend/tests/webhook-adapters.test.ts | Adds validator unit tests for adapter form rules. |
| docs/detectors/generic-webhook.md | Adds end-to-end operator walkthrough and troubleshooting for generic webhooks. |
| docs/configuration.md | Documents generic webhook adapter schema under correlation configuration. |
| docs/api.md | Documents new endpoint and response shape. |
| docs/adr/README.md | Adds ADR 020 to the index. |
| docs/adr/020-generic-webhook-adapter.md | Adds design rationale and decision record for the feature. |
| configs/correlation.yaml | Adds commented example webhook_adapters section. |
| ROADMAP.md | Marks generic adapter complete and notes follow-up transform functions. |
| README.md | Adds high-level mention and example of the generic adapter. |
| FEATURES.md | Lists the generic webhook adapter capability and link to docs. |
| Cargo.toml | Adds new deps: subtle and serde_json_path. |
| Cargo.lock | Locks new transitive dependencies. |
| CHANGELOG.md | Adds Unreleased entry for the generic webhook adapter. |
| AGENTS.md | Updates repo structure and API surface overview to include the new endpoint. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+158
to
+159
| | HTTP 401 with `"signature mismatch"` | Shared secret differs between sender and `RADWARE_WEBHOOK_SECRET`, or the sender hashed a different byte range (e.g. after `Content-Encoding: gzip`) — ensure HMAC is computed over the raw request body prefixd receives | | ||
| | HTTP 400 with `"victim_ip not found"` | JSONPath didn't match; verify with `jq` locally against a real payload | |
There was a problem hiding this comment.
The troubleshooting table doesn’t match the implemented behavior/messages: mapping failures like missing victim_ip return HTTP 200 with a per-event results[].status="error" (not a 400), and HMAC failures return an error like "HMAC signature verification failed" (not "signature mismatch"). Updating these symptoms to reflect the actual response codes/messages will prevent operator confusion.
Suggested change
| | HTTP 401 with `"signature mismatch"` | Shared secret differs between sender and `RADWARE_WEBHOOK_SECRET`, or the sender hashed a different byte range (e.g. after `Content-Encoding: gzip`) — ensure HMAC is computed over the raw request body prefixd receives | | |
| | HTTP 400 with `"victim_ip not found"` | JSONPath didn't match; verify with `jq` locally against a real payload | | |
| | HTTP 401 with `"HMAC signature verification failed"` | Shared secret differs between sender and `RADWARE_WEBHOOK_SECRET`, or the sender hashed a different byte range (e.g. after `Content-Encoding: gzip`) — ensure HMAC is computed over the raw request body prefixd receives | | |
| | HTTP 200 with per-event `results[].status="error"` and a message like `"victim_ip not found"` | JSONPath didn't match; verify with `jq` locally against a real payload | |
Comment on lines
+325
to
+328
| // Reset form when dialog opens with new adapter | ||
| useState(() => { | ||
| if (initial) setForm(initial) | ||
| }) |
There was a problem hiding this comment.
AdapterDialog is using useState(() => { ... }) to perform a side-effect (setForm(initial)), which runs during render and can trigger React warnings/loops (and won’t re-run when initial changes). Replace this with useEffect keyed on initial (or remove entirely since onOpenChange already resets the form) to avoid setting state during render.
Comment on lines
+329
to
+336
| let action = compiled | ||
| .action | ||
| .as_ref() | ||
| .and_then(|p| p.query(node).at_most_one().ok().flatten()) | ||
| .and_then(|v| v.as_str()) | ||
| .filter(|s| *s == "ban" || *s == "unban") | ||
| .map(|s| s.to_string()) | ||
| .unwrap_or_else(|| "ban".to_string()); |
There was a problem hiding this comment.
action mapping silently falls back to "ban" when the payload contains an invalid action value (anything other than "ban"/"unban"), which can cause unintended bans if a detector sends e.g. "resolved" or a misspelling. Consider treating an explicitly-present but unsupported action value as a per-event mapping error (and only default to "ban" when the action field is missing/null).
Suggested change
| let action = compiled | |
| .action | |
| .as_ref() | |
| .and_then(|p| p.query(node).at_most_one().ok().flatten()) | |
| .and_then(|v| v.as_str()) | |
| .filter(|s| *s == "ban" || *s == "unban") | |
| .map(|s| s.to_string()) | |
| .unwrap_or_else(|| "ban".to_string()); | |
| let action = match compiled | |
| .action | |
| .as_ref() | |
| .and_then(|p| p.query(node).at_most_one().ok().flatten()) | |
| { | |
| None | Some(Value::Null) => "ban".to_string(), | |
| Some(Value::String(s)) if s == "ban" || s == "unban" => s.clone(), | |
| Some(_) => return Err(MapError::Invalid("action")), | |
| }; |
Comment on lines
+253
to
+273
| let mut seen_names = std::collections::HashSet::new(); | ||
| for adapter in &self.webhook_adapters { | ||
| if !super::webhook::is_valid_name(&adapter.name) { | ||
| errors.push(format!( | ||
| "webhook_adapter '{}': name must match [a-z0-9-]{{1,64}}", | ||
| adapter.name | ||
| )); | ||
| } | ||
| if !seen_names.insert(adapter.name.clone()) { | ||
| errors.push(format!( | ||
| "webhook_adapter '{}': duplicate name", | ||
| adapter.name | ||
| )); | ||
| } | ||
| if let Err(e) = super::webhook::CompiledAdapter::compile(adapter) { | ||
| errors.push(format!( | ||
| "webhook_adapter '{}': invalid JSONPath: {}", | ||
| adapter.name, e | ||
| )); | ||
| } | ||
| } |
There was a problem hiding this comment.
CorrelationConfig::validate compiles JSONPaths but doesn’t validate other webhook adapter invariants that can break ingestion at runtime (e.g. confidence_scale <= 0, empty auth.secret_env/auth.header, or auth.algorithm not being sha256). Adding these checks here would surface misconfigurations as a 400 on config update/reload instead of 500s or surprising confidence behavior (division by zero → inf clamped to 1.0).
Comment on lines
+3
to
+36
| The generic webhook adapter lets you integrate any detector or telemetry system that can POST JSON, without writing Rust code. You declare one or more named adapters in `correlation.yaml`, and each becomes a fully-authenticated endpoint at `POST /v1/signals/webhook/{name}` that feeds events into the standard correlation and mitigation pipeline. | ||
|
|
||
| > For integrations that prefixd ships natively — Alertmanager, FastNetMon — use the dedicated endpoints (`/v1/signals/alertmanager`, `/v1/signals/fastnetmon`). The generic adapter is for everything else. | ||
|
|
||
| ## When to use it | ||
|
|
||
| - Commercial DDoS appliances (Radware DefensePro, NETSCOUT Arbor, A10 Thunder) | ||
| - Cloud alerting (GCP Cloud Armor, AWS Shield Advanced events, Cloudflare) | ||
| - Internal abuse / anomaly-detection platforms that emit JSON webhooks | ||
| - Quick prototyping of new signal sources before deciding whether they warrant a native adapter | ||
|
|
||
| See ADR 020 for design rationale. | ||
|
|
||
| ## End-to-end example: Radware DefensePro | ||
|
|
||
| This walkthrough integrates a hypothetical Radware DefensePro alert stream. Radware posts signed JSON with an HMAC-SHA256 signature in the `X-Signature-SHA256` header. | ||
|
|
||
| ### 1. Create the shared secret | ||
|
|
||
| Generate a random HMAC key, store it wherever you already store prefixd secrets (Docker Compose env file, Kubernetes `Secret`, systemd `EnvironmentFile`, etc.), and export it as the environment variable you'll reference from `correlation.yaml`: | ||
|
|
||
| ```bash | ||
| openssl rand -hex 32 # 64-character hex string | ||
| # export RADWARE_WEBHOOK_SECRET="<paste>" | ||
| ``` | ||
|
|
||
| The secret must be present at prefixd startup; it is read from the environment, **not** from YAML. The API will never return it. | ||
|
|
||
| ### 2. Add the adapter to `correlation.yaml` | ||
|
|
||
| ```yaml | ||
| correlation: | ||
| enabled: true | ||
| # … your existing correlation settings … |
There was a problem hiding this comment.
This doc says adapters are declared in correlation.yaml, but the example YAML is structured as a correlation: section (which matches prefixd.yaml, not standalone correlation.yaml). This is confusing given the code supports both; please clarify which file/layout the example targets (or show both formats).
- webhook.rs: reject explicit invalid 'action' values (e.g. "resolved", typos) with a new MapError::InvalidValue variant instead of silently defaulting to "ban". Missing/null still defaults to "ban". Adds two unit tests. - config.rs: validate() now rejects confidence_scale <= 0 / non-finite, empty auth.secret_env, empty auth.header, and any algorithm other than "sha256" so misconfiguration surfaces as a 400 on PUT/reload instead of runtime 500s or divide-by-zero confidence. Adds 6 unit tests. - webhook-adapters.tsx: replace the useState-as-side-effect with a proper useEffect keyed on `initial`. Fixes the React state-during-render anti-pattern and correctly resyncs the form when the adapter changes. - docs/detectors/generic-webhook.md: troubleshooting table now reflects actual response codes and messages (mapping failures return HTTP 200 with per-event results[].status="error"; HMAC failure uses "HMAC signature verification failed"). Step 2 now shows both the standalone correlation.yaml flat layout and the embedded prefixd.yaml layout.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a generic webhook adapter at
POST /v1/signals/webhook/{name}so any detector or telemetry system that can POST JSON can be integrated without touching Rust. Operators declare adapters in `correlation.yaml` with JSONPath field mappings, HMAC/bearer/none auth, and optional array batching via `root_path`.Design rationale in ADR 020; operator walkthrough in docs/detectors/generic-webhook.md.
Why
ADR 019 established dedicated Rust adapters per detector (Alertmanager, FastNetMon). That pattern doesn't scale to the long tail — commercial DDoS appliances (Radware, NETSCOUT, A10), cloud alerting (Cloud Armor, Shield), internal abuse systems. Operators shouldn't need to fork prefixd to wire in a new JSON detector.
What's new
Backend (`3b2d1c2`)
Frontend (`0c67874`)
Docs (`0c67874`)
Auth
Safety
Test results