configuration_aliases in child module terraform validate fails: Provider configuration not present

[RobertKeyser]

Terraform Version

v0.15.0

Terraform Configuration Files

terraform {
  required_version = ">= 0.15.0"
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      version               = "~> 3.0"
      configuration_aliases = [ aws.replica ]
    }
  }
}
...
resource "aws_kms_key" "replica_bucket_key" {
  provider = aws.replica
  ...
}
...

Expected Behavior

Expected to see valid configuration errors for any resource referencing the alias provider.

Actual Behavior

Errors on all resources using the alias provider. What I find interesting is that it says that the resources are in state, but there's no state, per terraform show.

PS C:\REDACTED\s3> terraform version
Terraform v0.15.0
on windows_amd64
+ provider registry.terraform.io/hashicorp/aws v3.37.0
PS C:\REDACTED\s3> terraform show       
No state.
PS C:\REDACTED\s3> terraform validate
╷
│ Error: Provider configuration not present
│
│ To work with aws_kms_alias.replica_bucket_key_alias its original provider configuration at provider["registry.terraform.io/hashicorp/aws"].replica is required, but it has been removed. This occurs when a provider configuration is removed while        
│ objects created by that provider still exist in the state. Re-add the provider configuration to destroy aws_kms_alias.replica_bucket_key_alias, after which you can remove the provider configuration again.

Steps to Reproduce

1. terraform init
2. terraform validate

Additional Context

This is a child module that I've migrated from v0.14.4. It was originally using proxy provider configuration. I tried running a validate on directly it after adding in the configuration_aliases setting. I'm able to run an apply on a main.tf that references it, but just not able to validate the child module.

References

[joe-a-t]

Is there any update on this issue? This issue is causing a similar, although much smaller scale, impact as #28803 in terms of cluttering up the plans that we ask engineers to review with warnings that are not material and we cannot do anything to resolve or silence.

In our case, we have a repo that contains our shared modules. That repo has a check that runs terraform validate on each of the modules. If we remove the empty provider block that is causing Warning: Empty provider configuration blocks are not required, then we are forced to remove the terraform validate check because it starts failing with Error: missing provider .... However, if we leave the empty provider block, we get a bunch of noise in the plans from that warning.

The preferred solution to the immediate issue would be changing terraform validate to behave the same with

terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      configuration_aliases = [ aws.foo ]
    }
  }
}

as it does with

provider "aws" {
  alias = "foo"
}

or at least spit out a warning from validate instead of an error since I'd rather have the warning get spit out on the validate that no one looks at unless if it fails than in the plan that gets run much more frequently and is reviewed by humans.

As far as taking a step back and thinking about how Terraform is used in the wild, could we revisit adding a flag to silence all warnings (eg Warning: Empty provider configuration blocks are not required) and notes (eg Note: Objects have changed outside of Terraform)? I completely get that the warnings and notes are helpful when debugging stuff and appreciate the effort that the Terraform team has put into exposing this information to users. However, this additional information is not relevant in all contexts and based on the comments I've seen in related issues, it looks like a lot of people are having issues with the amount of noise Terraform is currently generating and that is even breaking popular open source automation tools.

I'm happy to help contribute in any way I can to pushing this along as long as the PR will get reviewed.

[mkielar]

Could we not just add a -module flag to terraform validate, telling it to validate the code as if it's a reusable module, and not root module? This would make terraform stop worrying about missing provider configuration, and assume that the providers must be - well - provided, when the module is to be used. This way we could get rid of the "Empty provider configuration...." warning and still be able validate reusable modules in cases when we just don't have a root module...

[jbardin]

@mkielar, the issue here happens long before validate comes into play. In order to validate the config, the correct providers must be initialized. If the overall provider configuration is not correct, the configuration cannot be loaded at all (i.e. the error here is from loading the config, not from validate). The old behavior was incorrect for any version of terraform which could access namespaced providers, since there is no way to know that the correct provider is being used to obtain the schemas with which to validate the configuration.

For now, the only way to correctly validate a module which can accept providers as a parameter is to wrap it in a root module which defines in the required providers.

[mkielar]

@jbardin, obviously I don't know the internals of terraform too well. However, from a user stand point:

If I do following:

terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      configuration_aliases = [ aws.foo ]
    }
  }
}

I'm basically telling terraform that there will be two providers in this module: aws and aws.foo. However, if I just leave it at it, terraform validate fails with Error: missing provider provider["registry.terraform.io/hashicorp/aws"].foo.

If however I add:

provider "aws" {
  alias = "foo"
}

Then running terraform validate for such module passes without warnings. But why?! What's the difference? I didn't tell terraform anything more than it already knew! I just declared (just using different syntax) that there will be an extra aws.foo provider, but terraform already knew that from required_providers section, didn't it?

It seems to me, that in both cases terraform has all the information required to properly run validation and validate without errors, yet in the first case (with missing provider block) it somehow refuses to admit it ;). Seems like if we make terraform accept that truth, not even the -module flag would be needed.

[jbardin]

The provider block is not simply different syntax for the same thing. The required_providers block defined what providers are required by the module and what they will be called, while the provider block defines an actual configuration for a specific provider. Having a provider configuration declared within a module means we cannot expand that module into multiple instances, nor can that module later be removed from the configuration.

Older versions of terraform could treat the empty provider block as a "proxy" for a provider passed in, but there was no way to differentiate that from an actual provider declared within in the module in all cases. It was a confusing syntax overloading the meaning of the provider block causing it to change behavior based on the context of the parent module, and led to numerous issues and support escalations.

The primary reason an empty provider block in a module was not turned into an error was due to timing, with limited releases pending to fully deprecate the behavior before 1.0.

In order to test a non-root module in this way, something must always be added; either temporary provider configuration to make it validate as if it was a root module, or call the module from a dummy root module. How to best handle this is what needs to be designed here, while also planning on how to integrate any changes into the experimental test command.

[bendrucker]

Just published https://github.com/bendrucker/terraform-configuration-aliases-action to help with this. It generates provider blocks to satisfy all required configuration_aliases in the module. If you're looking to call terraform validate from Github Actions, you can just plop this step before run: terraform validate and validate your child module with required provider aliases as if it were a fully formed root module.