The whole point of Haiku is a holistic approach to system design, so no, you cannot easily replace the window manager. You can, however, write "decorator" plugins which draw different window borders, write shortcut plugins for "tiling", etc.

It indeed does not use GPU acceleration; Haiku as a whole does not have 3D drivers either (but there are plans.)