Feature request: All resources should export their import key as a computed attribute

[geekofalltrades]

This probably ends up being a feature request on terraform-plugin-sdk and/or terraform-plugin-framework, instead, but I figured I'd start it here.

I am using "import key" to refer to the value that can be used to import a resource, for lack of another name.

Current Terraform Version

$ terraform version
Terraform v1.0.6
on linux_amd64

Use-cases

I am undertaking a large-scale refactor of a monolithic root module, splitting it into multiple smaller root modules. To accomplish this without destroying and recreating huge swaths of resources, I am going to need to terraform state rm a lot of resources from the source module and terraform import them into the destination module(s).

As well-covered in issues like #19017 and #26364, import of existing infrastructure like this does not have first-class support in the declarative configuration. However, even the imperative terraform import, as a workaround, is completely unscriptable, because the import keys used for various resource types are not uniform and can't be determined programmatically. See, for example, aws_route53_record, whose import key is the worst that I've personally encountered.

Any script trying to orchestrate a mass import would need a huge switch statement containing special-case formatting for resources of certain types, written with the documentation for each resource type open alongside it in another window, because it's the only source of this information. Or, you would need to generate an importer from the source code of the provider plugin, where you have access to the ResourceImporter that actually defines the import key.

Attempted Solutions

A script which identifies the state to be moved from the source module to the dest module from a terraform state ls, runs terraform state show <line> on each of them to parse out the import key, runs terraform state rm to remove it from the source module, and runs terraform import to move it to the dest module.

The problem is that the information needed to form the import key can't be determined programmatically.

Proposal

Terraform plugin SDK(s) are enhanced to add an import_key computed attribute to resources. terraform state show would expose the value needed to import this resource into another state file, making mass moves between state files scriptable.

This is not a small-scale change, but is possibly more achievable in the short term than automated, declarative import of resources.

This could even provide a stopgap solution to import of resources not originally created by Terraform, if it were extended to not only resources, but data sources:

• Have some existing infrastructure.
• Write Terraform describing that infrastructure.
• Transition "resource" declarations in your new Terraform to "data" temporarily.

With this, Terraform could discover the resources to import, and then a script could loop over their import_keys to import them. This would admittedly be annoying to do at scale, but maybe better than nothing? Depending on your infrastructure, it could be easier to script the translation of resources into data sources than to script the translation of your infrastructure into varying formats of import key.

References

• Feature: Read before Create  #19017
• Import by declarative code over imperative command #26364

[apparentlymart]

Thanks for sharing this use-case, @geekofalltrades.

As you suggested, getting something like this working across all resource types in all providers would be a pretty heavy lift today because only the providers actually know their own conventions for import ids, and for the more complex ones (where it isn't just the id field verbatim) they tend to only have a parser for the id format, and not any code to generate it.

This unfolds from the fact that the original idea for terraform import was as a tool you only use as part of initially adopting Terraform, not something you use on an ongoing basis for other purposes such as your refactoring task here. However, I do agree that it would be good to have a way to "close the loop" here and fully automate this, if there's a reasonable way to get there.

Given that it seems like solving this would require a change to every resource type that supports import in all providers anyway, perhaps we'd be better served by a more explicit mechanism for providers to report the id you'd use to import something, rather than bundling it in as an attribute with a name that'd be just a convention rather than a specified protocol. I don't think that materially changes your proposal, because it would still require touching the SDK and all of the providers individually, but would potentially leave us in a better spot for implementing further automation around it.

On the other hand, for Terraform v1.1 we've been starting down the road of serving refactoring use-cases within configuration, starting with a configuration-based alternative to some of what we solve today with terraform state mv. The initial work doesn't try to address refactoring between configurations, but if we end up pursuing that in the near future then something like what we discussed in #26364, or hopefully something even more streamlined specifically for moving objects between configurations, might end up removing the need for what you proposed here. We don't have concrete plans for what's next yet, so I don't know if that's actually true, but I'm mentioning it for future reference in case a future reader on our team sees this pair of issues and wonders how they each relate to the work we've started.

A specialized solution for moving things between configurations without running the provider-specific import logic would be most ideal, because in principle it could be implemented without modifying providers at all; anything that involves changing every resource type across all providers will take many years to be fully useful, so a solution that cuts out that requirement would be preferable here I think.

Thanks again!

[geekofalltrades]

#27366 has been reopened, but the conversation is still locked. Based on

The only known work-around is to go to the documentation, determine the resource ID manually, then import it.

it sounds like there's overlap.