The part that I don't understand is why AMD/Intel/somebody else don't just implement at least the base CUDA for their products.

HIP is basically that, but they still make you jump through hoops to rename everything etc.

There are libraries written at a lower level that wouldn't be immediately portable, but surely that could be addressed over time as well.