Also the note at the bottom of that page: that they are doing away with AVX-512. A bit sad for those who need to squeeze maximum performance out of a CPU. (I gather it lives on in Xeon and Zen 4.)
AVX-512 is not supported on the CPUs with "Hybrid Technology", i.e. the current Alder Lake and Raptor Lake, which will be launched towards the end of 2022, to replace Alder Lake.
What will happen when the Gracemont small cores will be replaced at the end of 2023, is not known yet.
It is still possible that the successor of Gracemont will have AVX-512, in which case the Intel processors with "Hybrid Technology" will also have it, at that time.
Taking into account that Zen 4 is expected to have AVX-512, it is unlikely that Intel has not also planned for Meteor Lake (2023) to have it. Meteor Lake is supposed to be made using a more dense CMOS process than Alder Lake and Raptor Lake, which should enable Intel to implement AVX-512 in the small cores.
It seems silly to do that just due to asymmetric processors. The small cores could simply not support the instructions and the OS could then handle the fault by moving the task to a big core. Perhaps this could be signaled via a different CPU feature flag to avoid libraries using AVX512 instructions sporadically (e.g. in memcopy) and only use it in long-running loops. Or maybe give the OS a way to determine whether the CPU flags should be shown to a specific process or not. Or applications could install a SIGILL handler and deal with it in userspace.
This. And performance wouldn't be worse than using AVX twice...
But worst case scenario of the OS moving the process to a big core on an illegal instruction or scheduling it to the right core based on a required capabilities system would also be quite acceptable most of the time.
Plus, it'd be great for supporting more specialized cores designed for different purposes and running a single core ISA with extensions for their specific needs. IIRC, there are some ARM chips that have three different kinds of core.
That's a bold claim. What are your sources? The two latest things I worked on, JPEG XL and quicksort, see a 1.4 and 1.6x speedup from AVX-512. That's on SKX and includes the much-maligned throttling. On a system level, I doubt moving those to GPU is helpful.
Also, moving things to the GPU may be good for throughput but bad for latency depending on the workload, since offloading to GPU has a cost and data exchanges too.
> offloading to GPU has a cost and data exchanges too.
This is bad with dGPUs over the PCIe bus, but not so much with GPUs that share a very fast memory bus with the CPU. In this case, the layout of the data may prove challenging to keep the same for when you use a CPU and a GPU.
If you're processing a lot of data, you're better off moving it to the GPU. If you're processing only a little data, the speed-up doesn't matter. I wonder how wide is the Goldilocks Zone where AVX512 makes a practical difference?
This (2020) reports "Despite the fact that we send the entire data array to the video card and back, sorting on GPU of 800 MB of data is performed about 25-fold faster than on the processor."
Thanks for the example! Sounds like 1.6 GB/s on an entire Tesla K80 (300W TDP).
This is in fact several times slower than our results on Skylake (with half the TDP), but note that K80 is from 2014.
The "25-fold speedup", as is often the case for such reports, comes from not optimizing the CPU side.
Assuming AVX-512 actually works well (haven't had the opportunity to use it myself), it could be very useful in high-end gamedev. Data-oriented programming is popular there, which makes it much easier to optimize data transformations w/ SIMD. Good compilers can even do some auto-vectorization (it's a nice boost with no additional programmer effort but you can't rely on it much). GPUs tend to be already fully loaded by the intensive rendering workload, so there is a large incentive to squeeze as much as you can out of the CPU.
gpus generally aren't worth it for moderately cheap O(n) operations. avx 512 is really nice because if you match the memory speed, you beat GPU (since the cost of GPU will also be memory speed constrained).
'few workloads that benefit from AVX-512 would be better off on a GPU.'
Have you seen the state of GPU software development? GPUs are very expensive in cloud, are poorly supported in containers and virtual machines If you want to use GPU compute, some stuff is Nvidia-only, some stuff is glitching and crashy, it's probably not avaliable in your language of choice, etc.
It is literally impossible for me to add GPU compute to any of our corporate workloads, but I can tap into AVX easilly in my language of choice.
One would think that, but the reality is that there are some workloads which seem to fare better on AVX-512 currently. Usually where you can't get the benefit of the massive GPU parallelism/thread rotation into play, and you need low-latency between operations for a large-enough data set, while its still small enough for benefit from the cache on the CPU.
I'm pretty sure GPUs closes that gap over time though.
According to this article [0], x86_64 level 4 is AVX-512 itself. So no, you lose a level completely. Nothing gets borked, and you get level 3 hardware.
