Is this a 1-1 comparison? If the ARM compile is compiling to ARM binaries then there might be less work/optimizations since it is a newer architecture. Seems like a test with two variables that changed. Would be interesting to see them both cross-compile to their respective opposite archs.
Maybe not, but A) it's close-- most of the work of compiling is not microarchitecture-level optimizations or emitting code, and B) if you're a developer, even if some of the advantage is being on an architecture that it's easier to emit code for... that's still a benefit you realize.
It's worth noting that cross-compiling is definitely harder in many ways, because you can't always evaluate constant expressions easily at compile-time in the same way your runtime code will, etc, too, and have to jump through hoops.
If everything else is the same, that seems like a solid reason to prefer the ARM architecture even setting aside 1:1 comparisons. Isn't faster compilation and execution the whole point of a faster processor?