120 fps is a measure of throughput, not latency. A high frame rate is good, but it's no guarantee that you're not going to drop a frame. Dropping a frame kills the experience, especially in VR.
In any case, .Net (and I gather Java) games typically go to considerable trouble to reduce GC pressure by pre-allocating memory pools and reusing them during execution. For certain kinds of games, like where you load everything in a level up front and don't deallocate until you finish, that can be fine. For other kinds of games, that's a nasty constraint to try to deal with. It ends up bending your architecture in ugly ways that make it hard to develop the game.
