Hey, folks. Looking for some insight on distributed systems setups but not exactly sure how to phrase it. Here goes:
Does anyone know of any decent approaches to building organic systems of services that can be arbitrarily distributed? The two things that come to mind immediately are the BEAM and something like Kubernetes. I have no experience with Kubernetes, but it does seem a bit heavy handed. With the BEAM my concern would be platform buy-in for everything. I'm exploring ideas for a new system I'm planning, and I'd like to be able to mix and match different languages/platforms as time goes on, with a focus on building really small services that are composable and reusable. Is Kubernetes the best way to do this these days?
Things I'm looking for:
- No platform/language buy-in (each service/unit can do things however it needs to)
- Hot reload of services
- Ability to deploy multiple instances of a service
- Ability to deploy different versions of a service (e.g. version 1, version 1.1, version 2, etc.)
- Ability to manage/monitor all of said services
- Some sort of first class/idiomatic way to communicate between them (even if that's just http)
- Low operational overhead/complexity
Seems to me like the BEAM hits all of these except the first.
And Kubernetes hits all of these except the last xD
Thoughts?
P.S. Additionally I guess it would be nice to have some sort of super simple and versionable configuration that governs the current state of deployment. No idea what it would look like. Maybe something like terraform?
If you want to have a mostly BEAM system, but still have some flexibility here, you've got options.
a) beam for distribution and on every node; but for non-beam processing, beam supervises and sends the work through pipes/unix sockets/etc
b) dist for distribution, but some nodes are just 'C nodes'; OTP comes with a C library to connect to dist, which should be enough to be usable in most languages that can call into C libraries.
c) mostly beam and dist, but some nodes require other means to communicate. It's not as tidy, but if you need something, you need something. The real world is messy, whatever.