And my own stuff was so very limited and crufty, before I rather burned out on the yak shaving of it all. Though two requests for demos just this evening... sigh.
Perhaps picture the cylindrical "big screen" someone here mentioned. Run 3D programs like CAD in some of the windows. Control the window manger with keystrokes. Use a trackpad, a mouse, a 3D mouse. These inputs aren't tightly coupled to the rendered 3D space. Add a 6DOF controller. There's still no need to introduce a tight coupling - it can sit beside your mouse, or on your knee, and twitch. Discard the gaming objective of immersion, and show the outside world. Use a non-standard VR rendering stack, or AR, so that's performant. Open up the HMD horse blinders. Add some "comfort mode" tricks, and now the outside world is being used for balance, and the rendered world is freed to be bizarre. When your head moves, move the cylinder twice as fast, to reduce the need for head motion. Turn it with keys or mouse and no head motion. It's not a real cylinder - there's no reason when you turn it and turn it back, the same thing has to be there - it can be whatever you want, whatever is helpful. The motivation for a surface is coplanarity for optical scanning, and living in some hardware/wetware sweet spot. But one is a task-specific constraint, and both are local to the region currently shown, not global constraints. So punt the cylinder, and use an arbitrary aphysical topology. So now, while do you have a 3D visual display, and 3D HIDs, the contents and inputs are no more constrained to some imagined reality than in any usual window manager an apps. Choosing how much resemblance to reality to exhibit, as with scrolling "physics", is simply a UI design choice. ... Ah well - it's late - FWIW.
Not OP but the team at Leap Motion [1] has some very interesting demos. 3D UI's re-imagined from the ground up. Weirdly, because now everything is 3D, the best you can do is try to mimic actually 3D things perhaps, something that didn't work in 2D at all?
They're fun demos. And perhaps from the perspective of say mobile, they're kind of reasonable. Rotating your wrist? Waving your arm to press a button? Waving your arm to move a window? Not all that odd when holding a phone.
But from the perspective of laptops/desktops? What comes to my mind is: RSI; slowwwww, but maybe it's the touchpad version of a standing desk?; people use tiling window managers in part because waving a finger to place windows in 2D is already too burdensome.
I've just now failed again to find nice video, but if you've ever seen it, picture a professional artist tooling along on a Cintiq at insane speed. Stylus, and two handed multitouch flying. Zipping through enormous menus. Twitch gesturing through radial menus. Stylus input with pressure and tilt and rotation. Now imagine that interactive area was several centimeters high. And imagine transparent layers of content above the screen (you might sort of get a feel, by moving a semi-transparent window in front of another, and switch between reading one and the other, back and forth). Imagine a 3D visual environment extending below and above the screen. And around it, throughout the room. Now imagine your keyboard does multitouch, and instead of art, that's your code.
