Wrong STS endpoint for US Gov AWS regions in kustomize-controller 1.6.0

[LeRocque]

Hello, I recently upgraded to Big Bang 3.0.0 and this required flux to be upgraded to 2.6.1
The kustomize-controller started using the wrong AWS STS endpoint when the image was changed to 1.6.0
I am in GovCloud and I need the controller to use the Regional endpoint, but it is not.
I cannot get the endpoint to change no matter what I try.
I am able to curl the correct https://sts.us-gov-west-1.amazonaws.com endpoint from inside the container.
Please help if possible!

STS: AssumeRoleWithWebIdentity, https response error\n StatusCode: 0, RequestID: , request send failed, Post\n \"https://sts.arn.amazonaws.com/\": dial tcp: lookupts.arn.amazonaws.com on 10.100.0.10:53: no such host

[matheuscscp]

Hi @LeRocque we fixed a few related things in Flux 2.6.2 and 2.6.3, can you please upgrade to 2.6.3 and let me know if the issue persists?

[matheuscscp]

I assume this is because you are using AWS KMS for SOPS decryption with kustomize-controller?

[matheuscscp]

What's your current setup? Have you configured the kustomize-controller ServiceAccount with an AWS IAM Role? What does your SOPS configuration look like in the Secrets? Do you have the field .spec.decryption.secretRef set in the Kustomization? How does your Flux Kustomization YAML look like?

[matheuscscp]

What does the ARN of the KMS key look like? Please paste it here omitting the sensitive parts (e.g. AWS account)

[LeRocque]

I need to wait for newer images to be available on Ironbank before I can upgrade the controller past 1.6.0 unfortunately
I know that if I force it to use 1.5.1 I just get an incompatibility error because of the CRDs being upgraded for 1.6.0 so everything is still working correctly with the IRSA role and SOPS configuration. I changed nothing before upgrading to 1.6.0 images. Unfortunately I will be tied up with another issue for the rest of the day. I will check back here on Monday morning. Thank you for your quick responses!

[matheuscscp]

I found the issue, will be opening a PR soon

[matheuscscp]

@LeRocque The issue is fixed here and backported to the release series branch here.

I know this solution won't work for you since you depend on Ironbank, but in case anybody else affected needs a fix, they can build an image from the branch release/v1.6.x. Let's wait for Stefan to come back next week and we'll see about a Flux patch release or at least an rc image. Thanks for reporting!!

[LeRocque]

Awesome thank you @matheuscscp . I will be awaiting the Ironbank image update!

[matheuscscp]

@LeRocque Flux 2.6.4 is released with kustomize-controller 1.6.1:

https://github.com/fluxcd/flux2/releases/tag/v2.6.4

[LeRocque]

Kustomize-controller 1.6.1 made it's way into Ironbank and fixed the bug for me. Thanks again for your help @matheuscscp