Yeah the ML envs are insanely bloated, but honestly it works well enough for us doing that work. It's nice having lots of battery-included stuff. It really sucks having to manage the primary dependency chains (mostly centered around CUDA versions and GPU drivers), but it's slowly getting slightly better. The overall bloat of packages generally doesn't feel like it gets in my way.
I come from an embedded systems / microcontroller background though, so the bloat does feel completely insane.
For me things like Django feel "worse" overall in terms of developer QoL than the ML universe does, but maybe just because I don't know it as well and my last real webdev experience was before PHP4/HTML5.
I mean it's entirely fine, at least that's what the company decided.
The snark is mostly arising because I recently wrote a little tool to do an inventory of our private docker registry, because storage was getting difficult to manage. After a whole bunch of messy data collection and setup, it sets up a graph of the layers in the registry, assigns images and tags to owners. From there it backpropagates "ownership" of layers - if a layer is only referenced by images attributed to one team, that layer belongs to the team, else it's marked as mixed.
Before doing that, people were enthusiastic about making image builds more efficient, tagging smarter, collecting faster. A great atmosphere I'm finding myself enjoying more and more in the company, which is great.
However, doing the analysis pretty much showed us that 90% - 95% of the storage used in the private registry was used by these python/ml docker images. We found several projects whose CD images were not being garbage collected by the setup for 2 years or so. This wasn't great, but the entirety of their images took up less storage than like 2 base images of the ML stack.
It's the thing we have to do, but I reserve my right to make fun of it.
I come from an embedded systems / microcontroller background though, so the bloat does feel completely insane.
For me things like Django feel "worse" overall in terms of developer QoL than the ML universe does, but maybe just because I don't know it as well and my last real webdev experience was before PHP4/HTML5.