While FP64 is indeed important for supercomputers, the largest supercomputers have a great deal in common with AI infrastructure.

For example, high bandwidth, low latency interconnects, supporting GPU direct network messaging and IO, are important.

High memory bandwidth is also quite important.

Debugging and performance profiling at scale also commonly uses similar tools.

No they do not, because supercomputers have different partitions to cater different needs. For example, a supercomputer's half the nodes might lack a GPU to cater for the users which really need FP64 on CPU, and the other half will have GPUs for users which needs them. They will be served from different queues, so their jobs do not block each other.

OTOH, if you don't think nobody is focusing on FP64, look YoY performance gains on both CPUs and FPUs for high precision floating point performance. You'll be surprised.