Feature/export dlq drain tests - #1488
Merged
1nonlypiece merged 3 commits intoJul 30, 2026
Merged
Conversation
…ation - Add DLQ types (DlqEntry, DlqMetricsEvent, MetricsHook, FailureReason, DlqEventType) to src/services/exportQueue.ts - Implement addToDlq, configureDlq, resetDlq, getDlqEntries, getDlqEntry, getDlqDepth, requeueDlqEntry, discardDlqEntry, clearDlq - Wire addToDlq into processJob on permanent failure (retries exhausted) - PII sanitisation: userId/targetUserId replaced with SHA-256 opaque tokens - DLQ capped at configurable maxDlqSize (default 100), oldest evicted on overflow - MetricsHook fires on add/requeue/discard/clear; throwing hook caught + warn-logged - Structured log lines at warn (add) and info (requeue/discard/clear) — no raw PII - 42-test suite in src/tests/exportQueue.dlq-drain.test.ts covering all requirements: Req1 entry creation, Req2 PII sanitisation, Req3 query interface, Req4 drain operations, Req5 metrics hook isolation, Req6 observability logging - Edge cases: cap eviction, empty clear, discard-then-requeue, duplicate jobId, round-trip JSON serialisation - docs/export.md updated with full DLQ drain/metrics API reference Closes Disciplr-Org#626
- Add buildValidationError, flattenZodErrors, formatIssuePath, formatValidationError to src/lib/validation.ts — required by security.integration.test.ts and src/tests/validation.test.ts - Replace jest.spyOn on read-only ESM export (serializeExportData) with an alternative test strategy that achieves the same coverage without ESM hacks: use a 1-shot job (maxAttempts=1) to confirm DLQ entry is created, and verify the multi-attempt job is not in the DLQ (retryable path) - All 122 security integration tests and 42 DLQ drain tests now pass
|
@chinecherem58 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds DLQ drain operations and metrics-hook test coverage for
src/services/exportQueue.ts,closing Issue #626.
Changes
src/tests/exportQueue.dlq-drain.test.ts (new file)
42-test suite covering all requirements from
.kiro/specs/export-dlq/requirements.md:addToDlqcreates well-formedDlqEntry; failure classification(
serialization_error,data_fetch_error,unknown_error); DLQ cap eviction (oldest evictedwhen full);
processJobmoves job to DLQ after exhaustingmaxAttempts; retryable jobs areNOT added to DLQ
userIdandtargetUserIdreplaced with 8-char opaque SHA-256tokens; raw user IDs, Stellar addresses never appear in
DlqEntry; metrics hook receives onlysanitised data
getDlqEntries()returns newest-first snapshot; mutations toreturned array don't affect store;
getDlqEntry()returns entry orundefined;getDlqDepth()reflects current countrequeueDlqEntryreturnstrueand re-creates processablepending job; returns
falsefor unknown ID;discardDlqEntrypermanently removes entry;clearDlqremoves all and returns countdlq.entry_added,dlq.entry_requeued,dlq.entry_discarded,dlq.clearedwith correctdlqDepth; throwing hook is caught andwarn-logged without breaking the queue; no-hook mode works identically
requeue/discard/clear; all structured JSON
Edge cases: cap eviction, empty
clearDlq, discard-then-requeue same ID, duplicatejobId,JSON round-trip
docs/export.md
src/tests/exportQueue.dlq-drain.test.ts
jest.spyOnon read-only ESM export with an equivalent test strategy that avoidsthe ESM mutation restriction
Test Results
userId/targetUserId/ Stellar addresses never appear in anyemitted event or log line
Closes #626