Oh right, I get you. Yeah that could kinda work but it's putting a lot of faith in the accuracy of the hand tracking.

I suppose an ambitious implementation could try to fuse the hand tracking data with the IMU data from the Joycons for better resolution.