>With Cue, I personally think it is uglier than TOML and Dhall and only slightly less reader-ambiguous than YAML
I think you're missing the main point of these languages.
YAML, TOML, JSON are all basically the same thing modulo syntactic differences, which are largely a matter of taste.
Dhall, Cue, Jsonnet, BCL, etc. are in a different league - they allow you to express some computations with data in your config. It helps you to eliminate boilerplate and duplication in your config, avoid copy-paste mistakes, make configs more concise, express some complex abstractions specific to your domain. They're miniature programming languages for your configuration data.
If all you have is a few dozen key/value pairs to specify, it doesn't really matter which you choose, just a matter of taste. But as your config files suddenly span thousands LoC, duplicate same things over and over, need to configure dozens of different things (e.g. with jsonnet out of a single jsonnet source you can generate multiple configs for basically anything json/yaml/ini driven), get edited by multiple teams and you start looking for better ways to structure this mess, that's when these more advanced config languages come in to rescue your sanity.
>But then, why not use the language you're already using for the rest of your project, or a bash script to export some variables?
Well, for same reason, say, why people would use a javascript framework to build a webapp over vanilla js. Both could do the job, and for simple cases there's little reason to go with a framework resp. specialized config language.
But as your app/config gets larger and more complex, using a framework resp. config language would tend to get the job done more efficiently by providing you structure and toolbox with solutions to common pain points.
Config generators themselves tend to be a rather heavyweight all-or-nothing solution which leads people to compromise on some adhoc middle-ground solutions like YAML with jinja templates with unclear evaluation semantics. A good config language designed from the ground up can be so much better than this unholy yaml/jinja mess!
Finally, one of the key selling points of specifically Dhall is type checking. Implementing that in config generators in a generic untyped scripting language would be a nontrivial amount of boilerplate, and boilerplate elimination is what config languages are all about.
- Jsonnet (https://jsonnet.org/) - simpler syntax and less concepts to learn, just an extension of JSON. But no type checking. An open source offspring of Google's internal config language (GCL/BCL)
- Cue (https://github.com/cuelang/cue) - a more ambitious attempt to fix GCL/BCL by replacing inheritance as the fundamental compositional primitive with constraint unification.
We make heavy use of Jsonnet at work (https://databricks.com/blog/2017/06/26/declarative-infrastru...). It's worked great. Having a hermetic, pure templating system whose only output is a set of JSON/YAML files means that you can refactor fearlessly: as long as the materialized JSON/YAML doesn't change, you are 100% sure your refactor is safe. Bazel's StarLark dialect of Python (https://github.com/bazelbuild/starlark) has similar benefits.
There are odd corners in the language, but not something that most people will end up bumping into in typical usage. The templates certainly get messy in large configurations, but no more messy than any other code, and the hermeticity/purity greatly helps in managing the messiness. It's certainly less messy/odd than the copy-paste configs or be-spoke JSON/YAML templating systems that inevitably appear in messy deployment environments!
The last thing of note is the lack of static types: this definitely affects usability to some extent, and especially hinders IDE support from being as useful as it is in e.g. Java. But having a useful/ergonomic type system that fits this specific problem space is probably still an unsolved research question.
We tried to introduce Jsonnet at our org. It failed miserably because ops kept mistaking the name for JSON which they hated. (International multilingual team).
It was a real shame because ops then implemented some features of Jsonnet via scripts to to parse and merge YAML. What was 0 LOC in Jsonnet is now about 300 LOC plus custom CI checkers, all because of a marketing problem.
If I go to the mentioned Jsonnet homepage it says "A simple extension of JSON". The graphic explains its relation to JSON. The example looks awfully similar to JSON.
What I don't understand is the following: config files are read by text editors, and in the end, by human beings. Because of the latter they should have certain traits. We must agree on the importance of these traits before we can settle on a standard.
For me, important features are that they must be readable, and easily editable. They must be readable with a certain text editor (vi) for backwards compatibility. So that means it shouldn't require syntax highlighting or schema. Well, these 2 simple requirements of mine rule out anything remotely resembling JSON.
It just appears to me that JSON is for JavaScript developers, YAML for Python developers, and Dhall for ML (the whole family I suppose, not just Haskell) developers.
Well then if we're going that route then perhaps all we need is some kind of glue between text config and binary config (which reminds me of Systemd...). Ie. that it accepts multiple config file formats.
I'm a little confused by one part of what you wrote. You say that your config files must be readable by vi, and that in turn adds a no-highlighting-required constraint and a no-schema-required constraint, and that in turn adds a no-JSON-nor-anything-like-it constraint.
I deal with JSON all the time in vim, effortlessly. I'd be willing to deal with it in Notepad if necessary, and certainly in non-vim vi. Pipe it through a prettifier (lately I use `jq . file.json`) if necessary.
I don't need syntax highlighting, and I don't need a formalized schema (although I certainly appreciate an informal one, interpreted by the 1.0 Human Meatbrain I carry around). Also, if by "vi" you meant "vim", this is EVEN MORE confusing, because vim syntax-highlights JSON.
With vi I mean vi, not vim. If I meant vim, I'd have written vim. I use vim if its available (with my own configuration which includes syntax highlighting), but it isn't always available.
With my Human Meatbrain syntax highlighter I have far more issues with JSON than with say YAML or any other markup language.
Consider, for example, how easy the syntax is of a Wireguard configuration file. It is basically akin to a shell script or ini configuration file. And these have a proven track record. Why is that way of configuration broken in the first place? You could do things such as variables in shell scripts as well.
I also believe that the whole Systemd drama is basically because of moving away from such a proven track record. And it might very well be true that shell scripts are slow. That is why I argue for backwards compatibility and converting to/from formats. Which is something Dhall is able to (it can convert to/from YAML and JSON).
i'll throw in one of my projects as a contender: ytt - YAML templating tool - https://get-ytt.io (check out live playground!).
it works with yaml structures (hence avoids text templating problems) and uses familiar python-like language, starlark, making quite easy to get started. it makes use of yaml comments to assign metadata/templating directives to yaml nodes, so it looks something like this:
it doesn't include type checking, however, it does have a system to "overlay" structures on top of each other via overlay feature -- https://github.com/k14s/ytt/blob/master/docs/lang-ref-ytt-ov.... merge/replace/remove operations expect to find one node by default so map key typos or wrong structural nesting problems are caught easily in common cases.
There's also another competing json templating project from google in the works. Doesn't seem to be production ready yet, but it's open sourced already - https://github.com/cuelang/cue