If you've ever tried to dive into the NumPy build process you'd see why. It's unbelievably complicated... not that they really could do it better given that they are compiling about a billion scientific libraries and support alternatives and optimizations (like MKL).
Yes - unfortunately I have and I failed miserably.
These days I'm trying to see if there's a docker build that can build a great numpy (with all optimizations). Interestingly there are even docker images to call cuda APIs from python.