I was under the, evidently naive, understanding that there were some shenanigans done in the libc to pick optimal code paths based on the architecture. This would obviously have a startup cost on the first run, but presumably the dynamic linking would work to this advantage by having it in a fast state for most calls?
Not sure where I got that idea, though. :(. Will have to look into that later.
Even if libc does it, other packages wouldn’t. The compiler is what generates code and they generally don’t try to automatically generate for different instruction sets because they don’t know what code is what hotpath and worth optimizing for. You could probably try to build for multiple different CPU features at the top level and do a universal executable like Apple did for their CPU transitions but that’s a lot of expense to pay both in terms of compilation time AND in terms of size.
Not sure where I got that idea, though. :(. Will have to look into that later.