While memory usage is not a perfect metric of how "bloated" a piece of software is, I would't say it's useless either. That memory isn't just sitting there, it's being read, written, moved, meaning the software uses it to do things which also take up CPU and other resources. There is no escaping that doing more stuff in the background means less prioritization for your active usage so the software feels (and is) slow to respond.
Now if that background/automated stuff is useful to you then that's that, you can't have something for nothing. But if you don't care for most of it I don't see why you would keep it around and not benefit from a snappy system. If you've never tried using a window manager instead of Desktop Environment give it a try, it's extremely simple and you can have both in parallel. See if you miss anything from DEs, I for one don't.
Now if that background/automated stuff is useful to you then that's that, you can't have something for nothing. But if you don't care for most of it I don't see why you would keep it around and not benefit from a snappy system. If you've never tried using a window manager instead of Desktop Environment give it a try, it's extremely simple and you can have both in parallel. See if you miss anything from DEs, I for one don't.