It's good you asked, but I'm not ready to answer it in a useful way. It depends entirely on your use cases.
Some un-nuanced observations as starting points:
- Helm sucks, but so does Kustomize
- Cluster networking and security is annoying to set up
- Observability is awkward. Some things aren't exposed as cluster metrics/events, so you need to look at, say, service and pod state. It's not easy to see, e.g. how many times your app OOMed in the last hour.
- There's a lot of complexity you can avoid for a while, but eventually some "simple" use case will only be solvable that way, and now you're doing service meshes.
Maybe "wrong" is the wrong word, but there are spots that feel overkill, and spots that feel immature.
I'd argue that Kustomize is the bee's knees but editor support for it sucks (or, I'd also accept that the docs suck, and/or are missing a bazillion examples so us mere mortals could enlighten ourselves to what all nouns and verbs are supported in the damn thing)
> how many times your app OOMed in the last hour.
heh, I'd love to hear those "shell scripts are all I need" folks chime in on how they'd get metrics for such a thing :-D (or Nomad, for that matter)
That said, one of the other common themes in this discussion is how Kubernetes jams people up because there are a bazillion ways of doing anything, with wildly differing levels of "it just works" versus "someone's promo packet that was abandoned". Monitoring falls squarely in the bazillion-ways category, in that it for sure does not come batteries included but there are a lot of cool toys if one has the cluster headroom to install them
Some un-nuanced observations as starting points:
- Helm sucks, but so does Kustomize
- Cluster networking and security is annoying to set up
- Observability is awkward. Some things aren't exposed as cluster metrics/events, so you need to look at, say, service and pod state. It's not easy to see, e.g. how many times your app OOMed in the last hour.
- There's a lot of complexity you can avoid for a while, but eventually some "simple" use case will only be solvable that way, and now you're doing service meshes.
Maybe "wrong" is the wrong word, but there are spots that feel overkill, and spots that feel immature.