I don’t doubt that you’ve made Terraform work for your needs. The point isn't that Terraform can't be used for dynamic infrastructure—it's that doing so requires workarounds like pre-split state files, sequential applies, or additional tooling like Terragrunt. That’s not the same as having a truly dynamic system where resources can be created and modified in response to real-time conditions without external orchestration.
Terraform works well for many cases, but the fact that you had to “figure out how to do it properly” kind of proves the point—it’s not inherently designed for dynamically changing infrastructure within a single apply cycle. If it were, you wouldn’t need external coordination to handle something as simple as "create DNS records for all instances, even if the number changes at runtime."
At the end of the day, Terraform is great for declaring infrastructure, but it lacks the flexibility of programming infrastructure. If you’re happy with the trade-offs, great—but let’s not pretend those trade-offs don’t exist.
If your instances are created by Terraform itself, sure, you can use for_each with a data source to define DNS records dynamically. But if the instances are created dynamically outside of Terraform—such as through an auto-scaling group—then Terraform's static plan model becomes a problem.
Terraform data sources can read existing infrastructure, but they don't automatically trigger new resource creation based on real-time changes. That means your DNS records won't update unless you manually run terraform apply again, and they won’t be part of a single apply cycle. In contrast, a real programming language could handle this as a continuous process, responding to infrastructure changes in real-time.
So yes, you can query instances with a data source and use for_each—but unless you’re running Terraform repeatedly to catch changes, your DNS records won’t reflect real-time scaling events. That’s the exact limitation I’m talking about: Terraform isn’t imperative, it’s declarative, and it doesn’t react dynamically at runtime without external orchestration.
Terraform works well for many cases, but the fact that you had to “figure out how to do it properly” kind of proves the point—it’s not inherently designed for dynamically changing infrastructure within a single apply cycle. If it were, you wouldn’t need external coordination to handle something as simple as "create DNS records for all instances, even if the number changes at runtime."
At the end of the day, Terraform is great for declaring infrastructure, but it lacks the flexibility of programming infrastructure. If you’re happy with the trade-offs, great—but let’s not pretend those trade-offs don’t exist.