Skip to content

Add bulk send with pipelining to RabbitMQ.#41

Open
rjacobs31 wants to merge 6 commits intomasterfrom
feature/bulk-send
Open

Add bulk send with pipelining to RabbitMQ.#41
rjacobs31 wants to merge 6 commits intomasterfrom
feature/bulk-send

Conversation

@rjacobs31
Copy link
Contributor

@rjacobs31 rjacobs31 commented Jun 5, 2025

Ticket:
https://kalido.monday.com/boards/1192473331/pulses/1873297221

We need to increase throughput when sending events to RabbitMQ in bulk. The official recommendation is to use pipelining, by sending multiple messages to RabbitMQ without waiting for each one to complete before sending the next (and only then checking whether each was successful).

@rjacobs31 rjacobs31 requested review from HeinKalido and Copilot June 5, 2025 12:18
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a bulk send feature with pipelining for RabbitMQ to increase throughput when sending messages without waiting for individual acknowledgments. Key changes include:

  • Introduction of BulkSend and BulkSendTo API methods in the exchange implementation.
  • Updates to tests and mocks to support the new bulk sending functionality.
  • Minor dependency version bumps and code refactorings to support the changes.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/util_test.go Adds helper types and functions to support testing scenarios.
test/rabbit_events_test.go Updates test cases to verify cancellation and bulk send behavior.
test/auto_generated_rabbitGenMock.go Adds mocks for the new BulkSend and BulkSendTo methods.
rabbit.go Introduces the BulkSendMessageHandleFunc type.
go.mod Updates dependency versions.
exchange.go Implements the new bulk send methods and updates exchange declaration.
Comments suppressed due to low confidence (1)

exchange.go:441

  • [nitpick] Modifying the input messages slice in-place to mark successfully sent messages could lead to side effects if the same slice is reused elsewhere. Consider using a new slice for unsent messages to enhance code clarity and maintainability.
if ack { messages[i] = nil }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants