It's not about being perfect. It's about using the wrong tool for the problem.
Ultimately, code as configuration is the solution. The code generates the actual configuration-data which can be of an arbitrary format (e.g. json) since no one is gonna see it anyways except for debugging.
There are usually two arguments against that:
1.) code can be or become too complicated
It's a non-argument because while code can and will be complicated, the same configuration in non-code will be even worse. In the worst case, people will build things around the configuration because the configuration isn't sufficiently flexible.
What is true is that a single line of code is usually more complex than single line of yaml. The same can be said about assembly vs. any high level language. But it should be clear that what counts is the complexity of the overall solution, not a single line.
2.) code can be dangerous
This is a valid point. You will have to sandbox/restrict it and you need to have a timeout since it could run forever - and that creates some additional complexity.
I think this complexity is worth it in 90% of the cases. Even if you are not using code to generate the configuration, you probably still want to have a timeout, just in case of some malicious actor trying to use a config that takes forever to parse/execute etc.
But if you say that this is a nogo, then at least use a proper configuration language such as Dhall instead of YAML. It already exists. You don't need to invent a new configuration language to avoid YAML.
I think it's more of a progression thing. The config file starts out pretty simple, enough where one of the text file formats is clearly the right solution and code is over-complex. But then it grows and grows, continuing to accumulate more depth and use more features of the format, and maybe some hacky extensions. At some point, it gets complex enough that, if you designed it from the start with that feature set, being code would obviously be the right solution. But now the transition over is hard to pull off, so it tends to get put off for longer than it should be.
For instance looking at kubernetes, do you really think they didn't know in advance how complex things would get?
My explanation is different: often configuration files are used for things that are close to infrastructure/operations and those are done by folks who are already used to yaml and who are not used to high level coding as much. It's probably not a concious decision by them, it's just what they believe is the best.
K8S maintains the internal state of the system in JSON which is easy to convert to YAML which is seen as more readable in comparison. I think their choice to support JSON and YAML as an interface for configuring K8S is because of this.
I'd also guess that the expectation was that abstractions would form around YAML to make it more powerful. Helm uses Go templating language which supports logic and loops (but is very unpleasant to write complex configs with) and the operator pattern is also popular for more advanced cases. Ultimately both end up exposing some JSON/YAML interface for configuring, though. It is up to you to decide how you create that JSON/YAML.
They could have chosen Dhall as well, which also converts to json (or YAML). So I have to wonder: why did they choose YAML in the end?
> I'd also guess that the expectation was that abstractions would form around YAML to make it more powerful.
Yeah, now you have 3 layers: you have an arbitrary abstraction which is likely different for every tool, then you have yaml and then you have the internal json. That seems like a loss to me in terms of complexity.
Dhall was first released in 2018. Helm was started in 2015. Helm did flirt with using Jsonnet which would have been better, but I think they already had charts using templates.
Code also requires a runtime or needs to be compiled to something static, like YAML or JSON, in order to be consumed. That latter option is essentially how things are today given that templating tools the generate configurations are plentiful.
I think that this requirement makes broad adoption a lot more difficult for code-based configuration.
> That latter option is essentially how things are today given that templating tools the generate configurations are plentiful.
Yeah, that's what I mean partially by "building around the configuration". How is that better than writing the configuration directly in code?
Obviously, if the configuration is described with code then there is a runtime needed. But since the configuration is usually being run by program... there should be a runtime already. It's not like I'm saying that just any code/language should be available.
Ultimately, code as configuration is the solution. The code generates the actual configuration-data which can be of an arbitrary format (e.g. json) since no one is gonna see it anyways except for debugging.
There are usually two arguments against that:
1.) code can be or become too complicated
It's a non-argument because while code can and will be complicated, the same configuration in non-code will be even worse. In the worst case, people will build things around the configuration because the configuration isn't sufficiently flexible.
What is true is that a single line of code is usually more complex than single line of yaml. The same can be said about assembly vs. any high level language. But it should be clear that what counts is the complexity of the overall solution, not a single line.
2.) code can be dangerous
This is a valid point. You will have to sandbox/restrict it and you need to have a timeout since it could run forever - and that creates some additional complexity.
I think this complexity is worth it in 90% of the cases. Even if you are not using code to generate the configuration, you probably still want to have a timeout, just in case of some malicious actor trying to use a config that takes forever to parse/execute etc.
But if you say that this is a nogo, then at least use a proper configuration language such as Dhall instead of YAML. It already exists. You don't need to invent a new configuration language to avoid YAML.