Skip to content

Validate agent token for duplicate IP addresses #3673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

sjberman
Copy link
Collaborator

@sjberman sjberman commented Jul 30, 2025

Problem: In some environments, Pods could share an IP address (for example when a Job completes and a new Pod grabs that IP). This would cause problems when the nginx data plane Pod would attempt to connect to the control plane. The control plane would try to validate the token provided by the nginx agent, by using the IP address in the request to lookup the associated Pod. If multiple Pods existed with that IP address, the control plane would error out.

Solution: Fix the control plane logic to use more criteria when getting the proper Pod to verify the provided token.

Closes #3657

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked that all unit tests pass after adding my changes
  • I have updated necessary documentation
  • I have rebased my branch onto main
  • I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

Fix an issue where nginx Pod couldn't connect to control plane if multiple Pods shared the nginx Pod's IP address.

@github-actions github-actions bot added the bug Something isn't working label Jul 30, 2025
Copy link

codecov bot commented Jul 30, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.08%. Comparing base (e138d64) to head (85c46f8).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3673      +/-   ##
==========================================
+ Coverage   87.06%   87.08%   +0.01%     
==========================================
  Files         127      127              
  Lines       15602    15594       -8     
  Branches       62       62              
==========================================
- Hits        13584    13580       -4     
+ Misses       1859     1856       -3     
+ Partials      159      158       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Problem: In some environments, Pods could share an IP address (for example when a Job completes and a new Pod grabs that IP). This would cause problems when the nginx data plane Pod would attempt to connect to the control plane. The control plane would try to validate the token provided by the nginx agent, by using the IP address in the request to lookup the associated Pod. If multiple Pods existed with that IP address, the control plane would error out.

Solution: Fix the control plane logic to use more criteria when getting the proper Pod to verify the provided token.
@sjberman sjberman marked this pull request as ready for review July 31, 2025 14:36
@sjberman sjberman requested a review from a team as a code owner July 31, 2025 14:36
Comment on lines +165 to +167
LabelSelector: labels.Set(map[string]string{
controller.AppNameLabel: usernameItems[3],
}).AsSelector(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if, in this case, the job has the same labels, does that mean it would still error? is that just an edge case which we shouldn't need to worry about?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We check the Pod phase to ensure that only one pod is running, even if the Job fits this same selector

@bjee19
Copy link
Contributor

bjee19 commented Jul 31, 2025

@sjberman were you able to get in a situation like in the original issue and manually verify it works?

@sjberman
Copy link
Collaborator Author

@bjee19 Not quite, but I confirmed with the user what causes it, and the code avoids that scenario.

Copy link
Contributor

@bjee19 bjee19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, nice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working release-notes
Projects
Status: 🆕 New
Development

Successfully merging this pull request may close these issues.

Incorrect check of allocated IP addresses
2 participants