feat(observability): worker→container trace waterfall (W3C trace-context) - #143
Merged
Conversation
✅ Deploy Preview for lunorash ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (4)
📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughWorker dispatches now generate W3C trace context, forward it through shard and container requests, and include it in observability events. Runtime and container OTLP exporters reuse the context to preserve parent-child trace relationships. ChangesTrace context propagation
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Worker as dispatchSingleShard
participant Shard as ShardDO
participant Container as Container request
participant Sink as OTLP sink
Worker->>Worker: Generate traceId and spanId
Worker->>Shard: Forward traceparent
Shard->>Container: Propagate traceparent
Worker->>Sink: Emit event with trace context
Sink->>Sink: Reuse traceId and spanId
Container->>Container: Export span with parentSpanId
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
Thank you for following the naming conventions! 🙏 |
Contributor
|
Thank you for confirming the Contributor License Agreement! 🙏 |
prisis
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/do/src/shard-do.ts (1)
4630-4692: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winForward
traceparentinto batch re-entry.
buildBatchEntryRequest()should copy the outertraceparentheader too; otherwise/rpc-batchdrops trace context when it re-entersfetch(), while single/rpcrequests keep it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/do/src/shard-do.ts` around lines 4630 - 4692, Update buildBatchEntryRequest to copy the outer request’s traceparent header into each generated batch-entry request, preserving the existing header forwarding behavior so re-entered /rpc calls retain trace context.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@packages/do/src/shard-do.ts`:
- Around line 4630-4692: Update buildBatchEntryRequest to copy the outer
request’s traceparent header into each generated batch-entry request, preserving
the existing header forwarding behavior so re-entered /rpc calls retain trace
context.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 9766461f-69fe-4b62-a282-25abb7fae0ef
⛔ Files ignored due to path filters (2)
packages/codegen/__tests__/discover-containers.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/container/__tests__/client.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**
📒 Files selected for processing (3)
packages/codegen/src/emit.tspackages/container/src/client.tspackages/do/src/shard-do.ts
…titching
Phase 4 of the observability plan — the trace-correlation foundation. The
worker now generates one W3C trace context per RPC and both rides it on the
dispatch span and can hand it to a container, so container and worker spans
stitch into one trace instead of forming disconnected traces.
- shared: `buildTraceparent` / `parseTraceparent` W3C helpers in the shared
OTLP encoder both packages inline (`00-<32hex>-<16hex>-01`).
- runtime: generate `traceId`/`spanId` at single-shard dispatch entry; add
them to `ObservabilityEvent`; `otlpSink` reuses them for the span (instead
of minting random ids at emit time); and the runtime forwards them to the
shard as a `traceparent` header.
- container: `createContainerTelemetry({ traceparent })` (or the
`LUNORA_TRACEPARENT` env var) makes every span inherit the parent trace id
and hang off its span id; a missing/malformed value falls back to a fresh
root trace.
Both ENDS of the waterfall plus the wire format, fully unit-tested. The
remaining middle hop — the ShardDO reading the forwarded `traceparent` and
threading it into `ctx.containers`' outbound container fetch — is a follow-up
(it spans @lunora/do + @lunora/container and can only be validated against a
live worker+container).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018sRFb1136YE8KDmDbFMYmm
…ner fetches Completes the worker→container trace waterfall started in the previous commit. The trace context the worker forwards as a `traceparent` header now reaches the container's outbound fetch, so container spans stitch under the worker's trace end to end. - @lunora/do: the ShardDO reads `traceparent` off the inbound `/rpc` request into a `currentRequestTraceparent` field (cleared per request), exposed via a `getCurrentTraceparent()` getter — mirroring the `currentRequestIp` / `getCurrentIp` plumbing. - codegen: the generated `buildCtx` passes `this.getCurrentTraceparent()` to `createContainerContext`, so `ctx.containers` is trace-aware. - @lunora/container: `createContainerContext` threads the traceparent through the accessor chain (`get`/`any`/`pool` → `toRequest`), which stamps it on every outbound container request next to the target-port header. Unit-tested end to end at each seam (worker forwards, DO reads, ctx.containers injects, container otel consumes). The one thing only a live worker+container can exercise is the full round trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018sRFb1136YE8KDmDbFMYmm
prisis
force-pushed
the
feat/observability-trace-waterfall
branch
from
July 12, 2026 11:39
9d70f95 to
8ce9b4c
Compare
Accept a future `traceparent` version that appends trailing fields (parse the first four, per the spec) instead of dropping it, and reject the reserved `ff` version; validate the version and flags fields. Also document the per-request wiring a long-lived container needs to stitch each request's spans under the inbound `traceparent` header. Addresses thermo-nuclear review findings L1/L2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a drift guard asserting the app-logic files shared between examples/expo and templates/expo (Chat/Login UI + messages backend) stay byte-identical, so a forgotten copy-paste is a red CI run rather than a silently regressed example. Also rename the template's web-fallback `sessionStorage` local (localStorage-backed, shadowed the browser global) to `sessionStore`. Addresses thermo-nuclear review findings P1/L4. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
prisis
added a commit
that referenced
this pull request
Jul 21, 2026
The in-memory LogBuffer folded the seven ctx.log severities onto four tiers (trace→debug, log/info→info, fatal→error), so a line logged at trace or fatal was indistinguishable from debug or error in the Studio Logs panel — the two tiers #147 added were unreadable at the only place they surface locally. Buffer the level the caller actually logged at, widen the studio's mirrored LogLevel union to match, and source the panel's chip list + grouped summary from a new LOG_LEVEL_ORDER in the shared contract so the ordering has one home. Also drops a stale comment on otlpSink claiming trace correlation is "a later phase" — it shipped in #143 and the sink already reuses event.traceId/spanId. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
prisis
added a commit
that referenced
this pull request
Jul 21, 2026
…dispatch PR #143 propagated W3C trace context worker→shard→container and #147 stamped traceId/spanId onto every ctx.log line, but a handler still had no way to instrument a sub-operation: one SERVER span per dispatch was the entire worker trace surface. A slow request was one opaque bar. Adds ctx.trace(name, fn, attributes?) on Query/Mutation/ActionCtx. It returns the body's value unchanged and re-throws a failure after recording it as an error span — instrumentation, never flow control. Nesting is lexical, so the shape of the code is the shape of the waterfall. Design notes: - The span stack lives in a closure scoped to the ctx, NOT on `this` like the surrounding currentRequest* fields. Those are set-at-entry/cleared-at-exit and so are only sound across code that doesn't span an interleaving point, whereas a span stack is by definition held across the awaited body. Scoping it to the ctx makes concurrent dispatches structurally unable to corrupt each other's nesting. - The trace anchor IS resolved once per dispatch on `this`, so ctx.trace and the synthetic root span agree on the ids even with no inbound traceparent. - The root span is recorded only when the dispatch actually produced spans: minting one per request would fill the bounded ring with single-bar traces from uninstrumented handlers and evict the instrumented ones the panel exists for. It is not sent to onSpan — the runtime already emits the dispatch to onRpc, and a collector would otherwise show it twice. - foldTraces orders rows by (offset, depth), not arrival. Spans are recorded on completion, so a child is buffered before its parent, and at millisecond resolution the two routinely share a startTs — ordering has to come from the structure. The anchor is likewise picked structurally (the span whose parent is absent), since a fast parent/child pair is indistinguishable by timing. Caught by a test; both cases are now regression-covered. - The fold is total: the ring routinely holds partial traces (eviction can drop a parent, and a trace can be read before its root settles), so an orphan is re-parented onto the anchor rather than dropped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
This was referenced Jul 22, 2026
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 4 — Worker→container trace correlation (complete)
Final piece of the observability plan. A pure OSS framework change →
alpha(builds on the merged Phase 2 OTLP transport, #139). A worker's RPC span and the spans of any container the handler calls now stitch into one W3C trace, end to end.The waterfall, seam by seam
@lunora/runtime): atraceId/spanIdis minted at single-shard dispatch entry (before the shard runs the handler), added toObservabilityEvent, reused byotlpSinkfor the dispatch span (no more random-at-emit ids), and forwarded to the shard as atraceparentheader.@lunora/do): the ShardDO pullstraceparentoff the inbound/rpcrequest into a per-requestcurrentRequestTraceparentfield (cleared each request), exposed viagetCurrentTraceparent()— mirroring the existingcurrentRequestIp/getCurrentIpplumbing.ctx.containersbecomes trace-aware (@lunora/codegen): the generatedbuildCtxpassesthis.getCurrentTraceparent()intocreateContainerContext.@lunora/container):createContainerContextthreads the traceparent through the accessor chain (get/any/pool→toRequest), which stamps it on every outbound container request.@lunora/container/otel):createContainerTelemetry({ traceparent })(orLUNORA_TRACEPARENT) makes every span inherit the parent trace id and hang off its span id; a missing/malformed value falls back to a fresh root trace.shared (
shared/otlp.ts):buildTraceparent/parseTraceparentW3C helpers (00-<32hex>-<16hex>-01), inlined into both packages.Verification
@lunora/runtime500/500,@lunora/container135/135,@lunora/do1036/1036,@lunora/codegen763/763 (full suites — no regressions). New tests cover:otlpSinkreuses the event's ids; the container stitches under a parenttraceparent(and falls back on a malformed one);ctx.containersget/any/poolall stamp the forwarded traceparent; codegen wiresgetCurrentTraceparent()into the generated ctx.Backward compatibility
traceId/spanIdonObservabilityEventand the newcreateContainerContextargument are optional — existing sinks, dispatch paths, and generated code without containers are unaffected (otlpSinkfalls back to random ids; a container with notraceparentforms a fresh trace as before).🤖 Generated with Claude Code
https://claude.ai/code/session_018sRFb1136YE8KDmDbFMYmm
Summary by CodeRabbit
traceparentpropagation across worker dispatches, shard request handling, and container fetches to enable trace stitching.