perf(loggly): hoist per-request closure out of log phase#13648
perf(loggly): hoist per-request closure out of log phase#13648shreemaan-abhishek wants to merge 2 commits into
Conversation
The log phase reassigned the handle_http_payload closure on every request just to capture conf, allocating a new function object per request and adding GC pressure. Thread conf into handle_log directly and build the batch-processor handler once per config version instead of per request. This also removes a latent bug where the shared module-level closure could resolve the wrong conf when multiple routes use different loggly configs.
There was a problem hiding this comment.
Pull request overview
This PR reduces per-request overhead in the loggly plugin by removing a per-request function allocation in the log phase and ensuring the async batch handler uses the correct plugin conf when multiple routes use different loggly configurations.
Changes:
- Hoists the HTTP bulk-send logic into
handle_log(entries, conf)soconfis passed explicitly instead of captured via a per-request closure. - Removes the mutable module-level
handle_http_payloadindirection to avoid cross-route config mix-ups in async processing. - Binds
confonce per batch processor (via the batch handler closure passed tobatch_processor_manager:add_entry_to_new_processor) rather than once per request.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
membphis
left a comment
There was a problem hiding this comment.
[P2] Missing regression coverage for the cross-config wrong-conf bug
The code change looks right, but the PR claims to fix a bug where the module-level handle_http_payload could be overwritten by a later request from a different loggly configuration. The patch does not add a test that would fail on the old behavior: two routes/configs with different customer_token/tags, delayed batch flush, and interleaved requests.
Please add a Test::Nginx regression case that creates two distinct HTTP bulk loggly configs, sends interleaved requests before the first batch flushes, and asserts each batch is sent with its own token/tags. Without that coverage, this correctness fix can regress unnoticed.
Two http-bulk loggly configs with distinct tokens/tags buffer one entry each before either batch flushes; asserts each batch is sent to its own endpoint. Fails against the old shared module-level closure, which sent both batches with the last request's conf.
|
Added a regression test (
This fails on the previous behavior: the shared module-level |
Description
The
logglylogger reassigned thehandle_http_payloadclosure on every request inside_M.log(), solely to captureconf. Each request therefore allocated a fresh function object, adding avoidable GC pressure in the log phase. Benchmarking loggers showedlogglycarrying a disproportionately high per-request overhead compared to peers in the same class, and this per-request allocation was the cause (P50 is unaffected since the actual send happens in the async batch-processor timer).This PR hoists the handler to module scope:
confis threaded intohandle_log(entries, conf)directly, and the small binding closure is created once per batch processor (i.e. once per config version) rather than once per request. The separatehandle_http_payloadindirection is removed.As a side effect this also removes a latent correctness bug: the shared module-level
handle_http_payloadwas overwritten by whichever request ran_M.log()last, so with two routes using different loggly configs the async handler could send a batch with the wrongconf. Bindingconfper processor fixes that.Behavior is otherwise unchanged; existing
t/plugin/loggly.tcases (including TEST 15, the http bulk path that assertsconf.tags) continue to cover both the syslog and http paths.Which issue(s) this PR fixes:
Fixes #
Checklist