Use long polling in sqs CLI find_all so dump/mv don't stop early#1032
Use long polling in sqs CLI find_all so dump/mv don't stop early#1032mensfeld wants to merge 3 commits into
Conversation
find_all (used by sqs dump and sqs mv, both defaulting --number to Float::INFINITY) terminated on the first empty receive_message response. With the default short polling, SQS samples only a subset of its hosts and routinely returns an empty batch while the queue still has messages, so on real (distributed) SQS dump/mv quietly processed only a fraction of the queue (invisible on single-node LocalStack). Long poll (wait_time_seconds) and only stop after several consecutive empty batches, so the queue is actually drained. (No spec: exercising bin/cli in-process defines Shoryuken::CLI, and Shoryuken#server? is defined?(Shoryuken::CLI), which would flip server? globally and break other specs - same constraint as the CLI fix in the message_size PR.)
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthrough
Changesfind_all long-polling drain fix
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Integration spec (separate process, so loading bin/cli doesn't flip Shoryuken#server? for the unit suite) drives CLI::SQS#find_all with a scripted client containing a false-empty batch, and asserts it drains everything and long-polls - deterministic, since ElasticMQ never produces the false-empties this guards against. Plus a CHANGELOG entry.
find_all (used by sqs dump and sqs mv, both defaulting --number to Float::INFINITY) terminated on the first empty receive_message response. With the default short polling, SQS samples only a subset of its hosts and routinely returns an empty batch while the queue still has messages, so on real (distributed) SQS dump/mv quietly processed only a fraction of the queue (invisible on single-node LocalStack).
Long poll (wait_time_seconds) and only stop after several consecutive empty batches, so the queue is actually drained.
(No spec: exercising bin/cli in-process defines Shoryuken::CLI, and Shoryuken#server? is defined?(Shoryuken::CLI), which would flip server? globally and break other specs - same constraint as the CLI fix in the message_size PR.)
Summary by CodeRabbit
Bug Fixes
sqs dumpandmvso they continue through intermittent empty SQS responses and finish only after the queue is truly drained.Tests