This is one of the reasons why, coming from a background of writing Chef and writing exhaustive testing around my configuration management, I've never been able to deal with Ansible without grinding my teeth. Or, for that matter, Terraform; HCL is godawful and trying to write it in JSON is a sucker's bet, too. With the move towards more containerized systems I don't write Chef much anymore (thankfully), but Terraform has become ever more of a boil on my rear end. Fortunately there's also Pulumi now and writing this stuff in TypeScript is a lot faster and feels really good.

The thing is, although Ansible is not perfect, it is python. When you are writing playbooks you should be defining the state of the world, if you are writing custom logic it is probably best to write a custom module, which are commonly unit tested. I haven't tried Pulumi yet, thanks for pointing it out.

I'm not quite ready yet (esp. documentation is lacking), but I'm working on something to get the best of both worlds: https://freckles.io

Basically, you write your state-defining code in Ansible, using either modules, tasklists, roles, or a combination thereof. 'freckles' lets you wrap those up in re-usable, distinct, atomic units which you can combine for more complex tasks. Then you can use those directly via the command-line, or you can auto-generate (wrapper) Python (will do other languages later) classes from them (e.g. https://freckles.io/doc/interfaces/python#code ), for when there is more 'logic' to be implemented.

I reckon it's a bit like Pulumi, but it is less opinionated. Actually, it's probably more like it lets you create your own little domain-specific Pulumi, if that makes any sense. It also works really well as a wrapper for Packer and Terraform.

As I said, documentation is not quite there yet, but most of the important stuff works. Starting to look for people to try it out, if anybody on here is interested.

First off: congratulations on shipping something! Shipping is hard.

But this misses the mark. With respect, and I'm trying to not not dunking on your project, to me Ansible is the worst of every world and is emblematic of configuration management retreating from the realm of "infrastructure as code" to "infrastructure as magic notepad files because incurious sysadmins won't write code.". I don't really care that Ansible is being wrapped so long as, inevitably, I'm going to have to go deal with that when it breaks. Eventually Pulumi/Terraform will break too, and there is no programming language I have more eyerolls for than Golang, but at least it's a programming language, you know? (And I really don't relish the thought of ping-ponging from the Python wrapper to YAML hell to a Python module, tbh.)

The biggest thing that a project like Freckles fails to capture, and where Pulumi shines, is that it's all code and you just treat it as code. They're compiler wonks, or at least the guy I know there is one, and they leverage that--when dealing with stuff like Lambdas/GCP Cloud Functions, you just write them inline and they're hoisted into deployable packages without comment or incident. There's no zipping of files, there's no messing around--you wrote a function to do a thing and it gets run. Done-and-done.

It's got a lot of other nice features, but that it is transparently code, and that they're investing most of their effort (it seems) into hiding the unfortunate fact that eventually some Terraform providers get run, makes Pulumi a really hard one to top.

Fair enough, I can see some of your points, but (obviously) I don't think they are entirely relevant, and my experience and view of Ansible is a different one.

Also, Ansible is only one backend for freckles, it's just the first one I implemented because of all the roles and modules already available for it. There will be a Terraform backend, and a Packer one, and a Kubernetes one too, probably. And I'm actually much more excited about the shell backend I have in mind, but that is much more work to create a repository of tasks for. You'll be able to just select one of those backends, or combine all of them.

The goal is for 'end-users' never having to deal with underlying backends and Ansible code and the like, if they don't want to. Those are just there to provide the building blocks end users would use to compose their provisioning/orchestration -- which they can do entirely in code. I hope to have a stable 'stdlib' of such tasks at some point, comparable to what is available in Pulumi, which people can just use without any other concern.

I don't know how Pulumi works under the hood, are you saying they basically wrap Terraform modules? If that's the case, that's not that different to how freckles works. With the exception that freckles can also be used for non-cloud things.

Ansible is emphatically not Python. Ansible is YAML with Python extensions.

The model you're describing better fits Chef Zero. (Which is excellent, and is the tool I would go to were I still in a shop that needed instance-level CM.)

Ansible is written in Python, and all the core modules are written in Python. Using YAML you are passing arguments to the Python modules. This is also why Jinja is used.

https://docs.ansible.com/ansible/latest/dev_guide/index.html

Did it sound like I was unaware of this?

The user interface is YAML. It is an awful user interface and it adds nothing except complexity to the effort of making systems work. Expect people to write well-factored code and to treat the entirety of their CM as code and they will do so.