feat: support id token refresh inputs for backend and providers

[andrewhickman-aveva]

Implementation Plan

• Add a new flag to use native Azure DevOps capabilities in the providers and backend
• Set the ARM_OIDC_AZURE_SERVICE_CONNECTION_ID explicitly
• Update the azurerm backend settings / docs to match the updated docs and support URI lookup: https://developer.hashicorp.com/terraform/language/backend/azurerm

Original Issue Comment

Currently workload identity federation is supported by setting the oidc_token parameter

azure-pipelines-terraform/Tasks/TerraformTask/TerraformTaskV4/src/azure-terraform-command-handler.ts

Line 52 in d15902f

this.backendConfig.set('oidc_token', workloadIdentityFederationCredentials.idToken);

However this token expires after 10 minutes which can be problematic for long-running jobs. The azurerm provider has been updated to take a new set of parameters, oidc_request_url, oidc_request_token and ado_pipeline_service_connection_id. If all these are set, then terraform can request its own id token when necessary, fixing the issue for long-running jobs.

I'm not sure if this should be configurable, or if the task should just always use the newer parameter.

[mericstam]

Hi, Thanks for heads up.
@jaredfholgate is this something you can take a peek at and share some thoughts on?

br
Manuel

[jaredfholgate]

I plan to implement this once azuread has been released (just got merged).

It will then be available in azapi, azurerm, azuread and the azurerm backend giving us enough coverage to use.

My plans are to run some testing ASAP and then I'll submit a PR for this task. They will be set via env vars.

[jaredfholgate]

Version 5 is now released with ID token refresh as the default behaviour for Workload identity federation. We have no plans to back port this to v4 due to the complexity it would add to that version, so please try v5. Docs are also updated. Thanks for requesting this feature, well publish a blog post soon too.

[andrewhickman-aveva]

Thanks, I also like the switch to always use env variables in version 5

[nhart12]

With the v5 tasks I'm running into permission issues with WIF when running a terraform plan it can't load the state which is stored in blob

Error: error loading state: executing request: unexpected status 403 (403 This request is not authorized to perform this operation using this permission.) with AuthorizationPermissionMismatch: This request is not authorized to perform this operation using this permission.

I think this is due to us using different connections for the backend/init (backendServiceArm) vs the plan (environmentServiceNameAzureRM).. While this works in the v4 tasks, I think there might be an underlying difference in how ARM_OIDC_AZURE_SERVICE_CONNECTION_ID is handled by the azure_rm provider

[jaredfholgate]

With the v5 tasks I'm running into permission issues with WIF when running a terraform plan it can't load the state which is stored in blob

Error: error loading state: executing request: unexpected status 403 (403 This request is not authorized to perform this operation using this permission.) with AuthorizationPermissionMismatch: This request is not authorized to perform this operation using this permission.

I think this is due to us using different connections for the backend/init (backendServiceArm) vs the plan (environmentServiceNameAzureRM).. While this works in the v4 tasks, I think there might be an underlying difference in how ARM_OIDC_AZURE_SERVICE_CONNECTION_ID is handled by the azure_rm provider

Per the docs, any previous support for multiple identities was more by luck than design. It would only work when caching the creds (ID Token) in the plan file, which is not a reliable solution.

We added this note to the docs:

NOTE: Terraform on Azure does not support the use of separate credentials for backend storage account and the Azure providers at this time. This is because they share the same environment variable names. As such, the service connection used for the Terraform Task must have permissions on the storage account container for your backend state file, even if it is in a separate subscription.

However, I am looking to provide a solution to support this properly for WIF and have raised this PR as a starting point: hashicorp/terraform#36922

Essentially by adding alternative env vars to the backend, we should theoretically be able to support separate identities for backend and provider.

[nhart12]

Makes sense. Thanks @jaredfholgate I'll subscribe to that other issue