Yes, I understood your argument. However, your brain does not primarily use head movement to judge location. It primarily uses interaural time difference. Head movement may have some effect (your brain is a neural network trained to use all signals available to it), but ITD is the main factor, at least for most people.

Moreover, it's not clear that 4 point sources of audio can accurately reproduce real-world effects for the purpose of head movements, either.