User Managed Identity connection didn't work into azure devops with AzureCLI@2

[diegolagospagopa]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.6.6

AzureRM Provider Version

3.85.0

Affected Resource(s)/Data Source(s)

backend_state

Terraform Configuration Files

- task: AzureCLI@2
    displayName: "🔦 Plan Terraform"
    inputs:
      connectedServiceNameARM: '${{ parameters.AZURE_SERVICE_CONNECTION_NAME }}'
      addSpnToEnvironment: true
      scriptType: 'bash'
      scriptLocation: 'inlineScript'
      failOnStandardError: true
      workingDirectory: '${{ parameters.WORKINGDIR }}'
      inlineScript: |
        echo "##[section] 💈 Start terraform plan for tf-env-folder=${{ parameters.TF_ENVIRONMENT_FOLDER }}"

        export cliendId=$(az ad sp show --id ${servicePrincipalId} --query "appId" -o tsv) 
        echo "servicePrincipalId: ${servicePrincipalId}"
        echo "cliendId: ${cliendId}"
        
        export ARM_USE_MSI=true
        export ARM_CLIENT_ID=${cliendId}
        export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
        export ARM_TENANT_ID=$(az account show --query tenantId --output tsv)
        
        generate_state_output=""

        if [[ "${{ parameters.TF_SUMMARIZE }}" == "True" ]]; then
          echo "[info] Added option to generate tfplan"
          generate_state_output="-out=tfplan"
        else
          echo "📢 Summary not enabled: ${{ parameters.TF_SUMMARIZE }}"
        fi

        if [[ "${{ parameters.AKS_NAME }}" != "" ]]; then
          echo "[INFO] 🚀 Run terraform plan + kubernetes"
          ./terraform.sh plan ${{ parameters.TF_ENVIRONMENT_FOLDER }} -var k8s_kube_config_path_prefix="$(pwd)" -lock-timeout=300s -lock=false ${generate_state_output}
        else
          echo "[INFO] 🚀 Run terraform plan"
          ./terraform.sh plan ${{ parameters.TF_ENVIRONMENT_FOLDER }} -lock-timeout=300s -lock=false ${generate_state_output}
        fi

`

terraform.sh do:
1. az subscription set
2. terraform init
3. terraform apply

Debug Output/Panic Output

.

Expected Behaviour

The terraform is able to load the backend configuration, and continue with terraform init

Actual Behaviour

the terraform is unable to load the backend config, and can't start terraform init

Error: Failed to get existing workspaces: Error retrieving keys for Storage Account "pagopainfraterraformdev": azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/xyz/resourceGroups/my-infra-rg/providers/Microsoft.Storage/storageAccounts/pagopainfraterraformdev/listKeys?api-version=2021-01-01: StatusCode=400 -- Original Error: adal: Refresh request failed. Status Code = '400'. Response body: {"error":"invalid_request","error_description":"Identity not found"} Endpoint http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F

Error: Backend initialization required, please run "terraform init"

Reason: Initial configuration of the requested backend "azurerm"

The "backend" is the interface that Terraform uses to store state,
perform operations, etc. If this message is showing up, it means that the
Terraform configuration you're using is using a custom configuration for
the Terraform backend.

Changes to backend configurations require reinitialization. This allows
Terraform to set up the new configuration, copy existing state, etc. Please
run
"terraform init" with either the "-reconfigure" or "-migrate-state" flags
to
use the current configuration.

If the change reason above is incorrect, please verify your configuration
hasn't changed and try again. At this point, no changes to your existing
configuration or state have been made.

Steps to Reproduce

user managed identity:

1. create a managed identity with Contributor permission on the subscription

terraform:

1. setup the backend into a storage account
2. setup the env variables like this:

export ARM_USE_MSI=true
export ARM_CLIENT_ID=${cliendId}
export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
export ARM_TENANT_ID=$(az account show --query tenantId --output tsv)

that allow to load the managed identity
3. run terraform init

Important Factoids

No response

References

• multiple msi support for azurerm remote state #27923
• Identity not found, when deployed from Azure devops arkiaconsulting/terraform-provider-akc#5

The project iac can be found here:

• https://github.com/pagopa/pagopa-azure-devops/tree/iac-core-migration/azure-devops/iac

[rcskosir]

Thanks for taking the time to open this issue. Issues related to the Azure backend live in the hashicorp/terraform repo, so I am going to move this issue there.

[diegolagospagopa]

Azurerm service connections with federated managed identity manual create a user managed identity that federates with azdo via oidc (Configure an identity managed by an external OpenID Connect Provider to access Azure resources as this application)

Currently therefore you need to use this code snippet to allow the connection correctly

  - task: AzureCLI@2
    displayName: "🔦 Plan Terraform"
    inputs:
      connectedServiceNameARM: '${{ parameters.AZURE_SERVICE_CONNECTION_NAME }}'
      addSpnToEnvironment: true
      scriptType: 'bash'
      scriptLocation: 'inlineScript'
      failOnStandardError: true
      workingDirectory: '${{ parameters.WORKINGDIR }}'
      inlineScript: |
        echo "##[section] 💈 Start terraform plan for tf-env-folder=${{ parameters.TF_ENVIRONMENT_FOLDER }}"

        export cliendId=$(az ad sp show --id ${servicePrincipalId} --query "appId" -o tsv)

        echo "[INFO] Terraform setup variables"
        export ARM_USE_OIDC=true
        export ARM_OIDC_TOKEN=${idToken}
        export ARM_CLIENT_ID=${cliendId}
        export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
        export ARM_TENANT_ID=$(az account show --query tenantId --output tsv)

        generate_state_output=""

        if [[ "${{ parameters.TF_SUMMARIZE }}" == "True" ]]; then
          echo "[info] Added option to generate tfplan"
          generate_state_output="-out=tfplan"
        else
          echo "📢 Summary not enabled: ${{ parameters.TF_SUMMARIZE }}"
        fi

        if [[ "${{ parameters.AKS_NAME }}" != "" ]]; then
          echo "[INFO] 🚀 Run terraform plan + kubernetes"
          ./terraform.sh plan ${{ parameters.TF_ENVIRONMENT_FOLDER }} -var k8s_kube_config_path_prefix="$(pwd)" -lock-timeout=300s -lock=false ${generate_state_output}
        else
          echo "[INFO] 🚀 Run terraform plan"
          ./terraform.sh plan ${{ parameters.TF_ENVIRONMENT_FOLDER }} -lock-timeout=300s -lock=false ${generate_state_output}
        fi