Skip to content

Add DescribeLogGroups/PutRetentionPolicy rate limiter #1695

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

duhminick
Copy link
Contributor

@duhminick duhminick commented May 19, 2025

Description of the issue

DescribeLogGroups (DLG) and PutRetentionPolicy (PRP) are getting throttled when there are either high deployment of agents and/or high number of log groups configured.

Description of changes

Adding a rate limiter for both DLG and PRP that limits to one operation call per second.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

  1. The test is just like the other tests except the time between calls are recorded. The assertion is that the time between calls is greater than or equal to 995 milliseconds. Ideally this would just be 1 second, but the time between calls is not exactly perfect so I had loosened up the threshold.
    a. This test is done for both DLG and PRP

You may notice that a mutex is used for the call times array, that's because the data-race-check would fail since multiple go routines are writing to that array.

Requirements

Before commit the code, please do the following steps.

  1. Run make fmt and make fmt-sh
  2. Run make lint

@duhminick duhminick changed the title add dlg and prp rate limiter Add DescribeLogGroups/PutRetentionPolicy rate limiter May 19, 2025
@duhminick duhminick marked this pull request as ready for review May 19, 2025 21:15
@duhminick duhminick requested a review from a team as a code owner May 19, 2025 21:15
Copy link
Contributor

This PR was marked stale due to lack of activity.

@github-actions github-actions bot added the Stale label May 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant