Except the whole world isn't really competing for those Visas. The appeal of the H1B is the difference in quality of life between the home country and the US. It's not like the everyone in the world has a burning desire to pay California rent.