feat(runtime,do): head + error-biased trace sampling - #166
Conversation
Add a configurable trace-sampling policy to the observability pipeline, mirroring Cloudflare Workers' head_sampling_rate. A deterministic per-trace head decision (derived from the traceId) keeps or drops a whole trace, biased to always keep traces that produced an error span (tail bias). Only trace spans are governed; metrics and logs are untouched. - shared/sampling.ts: zero-dep deterministic helpers (traceIdToUnitInterval, isTraceHeadSampled, resolveTraceSampling, shouldExportTrace). - shared/otlp.ts: buildTraceparent encodes the sampled flag; parseTraceparent returns it, so the runtime propagates its head decision to shards. - runtime: CreateWorkerOptions.sampling; emitRpcEvent gates the SERVER span; dispatchSingleShard sets the traceparent flag + x-lunora-sample-errors header. - do: ShardDO reads the propagated flags, holds sampled-out ctx.trace spans out of the live export, and re-decides at the dispatch finally (export on error). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
✅ Deploy Preview for lunorash ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (3)
📒 Files selected for processing (1)
Walkthrough
ChangesTrace sampling and tail-biased export
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Worker
participant ShardDO
participant ObservabilitySink
Worker->>Worker: Resolve trace sampling decision
Worker->>ShardDO: Dispatch with traceparent and error-retention header
ShardDO->>ShardDO: Buffer spans and track dispatch outcome
Worker->>ObservabilitySink: Emit sampled RPC event
ShardDO->>ObservabilitySink: Flush retained error-trace spans
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Thank you for following the naming conventions! 🙏 |
|
Thank you for confirming the Contributor License Agreement! 🙏 |
Merging this PR will not alter performance
Comparing Footnotes
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/do/src/shard-do.ts`:
- Around line 4847-4909: The sampled-out trace state in recordSpan and
flushSampledOutTrace is stored in shared per-instance fields, allowing
interleaved dispatches to overwrite sampling and sink decisions. Snapshot the
sampling decision, trace identity, and sink within each dispatch’s tracer/anchor
context (leveraging makeTracer’s per-call sink closure), or key deferred
held-trace state by traceId, so recordSpan and flushSampledOutTrace use the
originating dispatch’s state and preserve error flushing under concurrent
dispatches.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 650385eb-7039-4b03-8b4d-c06a98f348bb
⛔ Files ignored due to path filters (4)
packages/do/__tests__/shard-do.sampling.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/sampling.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**shared/otlp.tsis excluded by none and included by noneshared/sampling.tsis excluded by none and included by none
📒 Files selected for processing (4)
packages/do/src/shard-do.tspackages/runtime/src/create-worker.tspackages/runtime/src/index.tspackages/runtime/src/observability.ts
Records the new TraceSamplingConfig interface, the `sampling` worker option, and the emitRpcEvent sampling arg in the runtime/lunora api-snapshots. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
…ispatches CodeRabbit (#166, Major): the sampled-out / keep-errors / held-sink state was stored in flat per-instance fields (`currentTraceSampled`, `currentTraceKeepErrors`, `currentDispatchSink`) and read in recordSpan / flushSampledOutTrace, which run after a span body or the whole dispatch settles. A Durable Object interleaves dispatches across await points, so a sibling dispatch's `finally` could reset those fields first — leaking a sampled-out trace's spans live, or (worse) dropping a sampled-out+errored trace's held spans and defeating the always-sample-errors tail bias. The `currentRequestTrace` identity check in recordSpan had the same staleness. Replace the three fields with a single `traceSampling` Map keyed by traceId, registered at dispatch entry and deleted in the `finally`; recordSpan looks up by the span's own `traceId` and flushSampledOutTrace by the dispatch's trace, so two concurrent dispatches can't clobber each other's verdict or sink. Adds an interleaved-dispatch regression test (a slow sampled-out error trace whose flush runs after a sibling sampled-in dispatch's finally). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
From better-auth-ui #195, #193, #169, #166, #74 and #337. #195 — the organization limits are now published by uiConfig(). better-auth enforces organizationLimit / allowUserToCreateOrganization server-side and exposes no endpoint that reports them, so a UI can only find out by being refused. The create form is disabled with the reason instead. A function-form allowUserToCreateOrganization is a per-request decision the endpoint cannot evaluate, so it reports allowed and lets the server refuse — the alternative hides the button from everyone because one user might be denied. #166 — the 2FA card no longer vanishes for an OAuth-only account; it explains that a password is needed first. A setting that is simply absent reads as 'this app doesn't support 2FA'. Only a successful accounts read that found no credential row counts: while loading, or on error, the card behaves as before — the flow gate's rule, don't hide what you cannot reason about. #74 — the TOTP key is shown beside the otpauth:// URI. Most authenticators reject a pasted URI, so a desktop app or a broken camera had no way in. #193 — createActiveMemberController resolves the signed-in user's role in the active organization, which otherwise means fetching the org and finding yourself in members at every call site. A rendering hint, not authorization. #337 — organization.showSlug hides the slug field; create already derives one from the name. #169 was already supported by that same fallback. #165 (organization custom fields) is not done: it needs an additionalFields design that spans every card, not a flag. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tier 0 — trace sampling (mirrors Cloudflare
head_sampling_rate)Configurable head sampling with an always-keep-errors tail bias, so high-traffic apps cut trace volume without losing error traces.
CreateWorkerOptions.sampling = { headRate?: 0..1 (default 1), alwaysSampleErrors?: boolean (default true) }.traceId(shared/sampling.ts, zero-dep) → a trace is wholly kept or dropped (no half traces).emitRpcEvent;ctx.traceINTERNAL spans gated in the DO — the head decision propagates via the W3Ctraceparentsampled flag + anx-lunora-sample-errorsheader (no DO codegen config needed). Spans always buffer locally (Studio panel unaffected); the export decision issampled || (alwaysSampleErrors && traceErrored), made at the flush choke point so the error-keep works even when the error is only known at the end.sampling⇒ everything exported as before.Verify: runtime 585 / do 1158 tests pass; tsc + eslint clean; container
otelconsumers still pass.🤖 Generated with Claude Code
Summary by CodeRabbit