The Wii might not actually be able to detect the remote's exact location and orientation in 3D space. I'm not sure if Nintendo actually made the claim that it could, but that's certainly what I was led to believe.
Then again you'd need an extra sensor or two placed on the body somewhere to give the wii an idea of where the controller was in relation to the rest of your body. Unless it assumed your arm was in a specified position in relation to the rest of your body most of the time.
Oh well. I think they might've taken the best route for the goal they had in mind.