-
Notifications
You must be signed in to change notification settings - Fork 106
Description
We cannot use azwi to create either a service account or a federated identity in the AzureUSGovernment cloud. The endpoint that is being hit to look up our subscription-id is the public endpoint, not the AzureUSGovernment endpoint and thus when our subscription-id is queried it returns a 404. When using --azure-env
to set it to AzureUSGovernment or AzureUSGovernmentCloud, the setting is ignored.
Steps To Reproduce
azwi serviceaccount create phase sa \
--aad-application-name "${APPLICATION_NAME}" \
--service-account-namespace "${SERVICE_ACCOUNT_NAMESPACE}" \
--service-account-name "${SERVICE_ACCOUNT_NAME}" \
--subscription-id "${SUBSCRIPTION_ID}"
and
azwi serviceaccount create phase federated-identity \
--aad-application-name "${APPLICATION_NAME}" \
--service-account-namespace "${SERVICE_ACCOUNT_NAMESPACE}" \
--service-account-name "${SERVICE_ACCOUNT_NAME}" \
--service-account-issuer-url "${SERVICE_ACCOUNT_ISSUER}"
--subscription-id "${SUBSCRIPTION_ID}"
Expected behavior
Based on the documentation found here the following output or some version thereof should be generated, but it doesn't work.
INFO[0000] No subscription provided, using selected subscription from Azure CLI: REDACTED
INFO[0032] [federated-identity] added federated credential objectID=REDACTED subject="system:serviceaccount:default:workload-identity-sa"
Logs
First is for attempting to create a service account:
Mon, 25 Mar 2024 15:33:24 EDT cloud/azureclient.go:202 Resolving tenantID {"subscriptionID": xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"}
Mon, 25 Mar 2024 15:33:24 EDT transport/round_trippers.go:466 curl -v -XGET -H "Accept: application/json" -H "User-Agent: azsdk-go-armsubscriptions/v1.1.1 (go1.21.7; Windows_NT)" -H "Authorization: Bearer"
'https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx?api-version=2021-01-01'
Mon, 25 Mar 2024 15:33:24 EDT transport/round_trippers.go:495 HTTP Trace: DNS Lookup for management.azure.com resolved to [{4.150.240.10 }]
Mon, 25 Mar 2024 15:33:24 EDT transport/round_trippers.go:510 HTTP Trace: Dial to tcp:4.150.240.10:443 succeed
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:553 GET https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx?api-version=2021-01-01 404 Not Found in 411 milliseconds
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:570 HTTP Statistics: DNSLookup 85 ms Dial 48 ms TLSHandshake 216 ms ServerProcessing 60 ms Duration 411 ms
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:577 Response Headers:
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 Pragma: no-cache
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 X-Ms-Request-Id: 4d1a05d1-1547-4127-8786-f96c7d144251
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 Strict-Transport-Security: max-age=31536000; includeSubDomains
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 Cache-Control: no-cache
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 Expires: -1
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 X-Ms-Correlation-Request-Id: 4d1a05d1-1547-4127-8786-f96c7d144251
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 X-Content-Type-Options: nosniff
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 Content-Type: application/json; charset=utf-8
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 X-Ms-Failure-Cause: gateway
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 Date: Mon, 25 Mar 2024 19:33:25 GMT
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 Content-Length: 129
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 X-Ms-Routing-Request-Id: EASTUS:20240325T193325Z:4d1a05d1-1547-4127-8786-f96c7d144251
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 X-Cache: CONFIG_NOCACHE
Mon, 25 Mar 2024 15:33:25 EDT transport/round_trippers.go:580 X-Msedge-Ref: Ref A: 97520412180E4B30AC98BCA191A30161 Ref B: MNZ221060618031 Ref C: 2024-03-25T19:33:25Z
Error: header "WWW-Authenticate" not found in get subscription response
The next is attempting to create a federated identity:
Tue, 26 Mar 2024 10:42:34 EDT cloud/azureclient.go:202 Resolving tenantID {"subscriptionID": "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx"}
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:466 curl -v -XGET -H "Accept: application/json" -H "User-Agent: azsdk-go-armsubscriptions/v1.1.1 (go1.21.7; Windows_NT)" -H "Authorization: Bearer"
'https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx?api-version=2021-01-01'
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:495 HTTP Trace: DNS Lookup for management.azure.com resolved to [{4.150.240.10 }]
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:510 HTTP Trace: Dial to tcp:4.150.240.10:443 succeed
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:553 GET https://management.azure.com/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx?api-version=2021-01-01 404 Not Found in 347 milliseconds
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:570 HTTP Statistics: DNSLookup 31 ms Dial 46 ms TLSHandshake 197 ms ServerProcessing 70 ms Duration 347 ms
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:577 Response Headers:
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 X-Ms-Failure-Cause: gateway
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 X-Ms-Correlation-Request-Id: 2894f6bf-6f8a-4d28-998d-8e7c0ec14e8b
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 X-Content-Type-Options: nosniff
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 Cache-Control: no-cache
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 X-Ms-Request-Id: 2894f6bf-6f8a-4d28-998d-8e7c0ec14e8b
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 Date: Tue, 26 Mar 2024 14:42:33 GMT
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 Pragma: no-cache
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 Expires: -1
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 X-Ms-Routing-Request-Id: EASTUS2:20240326T144234Z:2894f6bf-6f8a-4d28-998d-8e7c0ec14e8b
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 Strict-Transport-Security: max-age=31536000; includeSubDomains
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 X-Msedge-Ref: Ref A: 74A19B4E725345C18A15E8879D842679 Ref B: MNZ221060609025 Ref C: 2024-03-26T14:42:34Z
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 Content-Type: application/json; charset=utf-8
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 X-Cache: CONFIG_NOCACHE
Tue, 26 Mar 2024 10:42:34 EDT transport/round_trippers.go:580 Content-Length: 129
Error: header "WWW-Authenticate" not found in get subscription response
Environment
Azure USGoverment GCC High IL-4
- Kubernetes version (use
kubectl version
):
Client Version: v1.28.4
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.26.10
- Cloud provider or hardware configuration: AzureGov using AKS
- OS (e.g:
cat /etc/os-release
):
PRETTY_NAME="Ubuntu 23.04"
NAME="Ubuntu"
VERSION_ID="23.04"
VERSION="23.04 (Lunar Lobster)"
VERSION_CODENAME=lunar
NOTE: This also doesn't work with Windows 10 using azwi for powershell either.
- Kernel (e.g.
uname -a
):Linux <HOSTNAME> 5.4.72-microsoft-standard-WSL2 #1 SMP Wed Oct 28 23:40:43 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
- Install tools: I installed this by grabbing the release from github:
wget https://github.com/Azure/azure-workload-identity/releases/download/v1.2.1/azwi-v1.2.1-linux-amd64.tar.gz
and then moving to/usr/bin/azwi
Additional context
When attempting to run the commands on the guide we can tell that it's reaching out to the wrong endpoint, hence the 404 when looking up our subscription-id. We are attempting to set this up in order to support the usage of external secrets operator by following their instructions to use azwi. I noticed there's a ticket open already about it not working in non-public clouds (#566 ) We are also not sure why Error: header "WWW-Authenticate" not found in get subscription response
is being returned in the logs, but since it doesn't match the anticipated outcome in the guide, we presume it's due to the cloud API endpoint being incorrect.