If there was a rpi hat that had multiple connections I would probably use it, a $300 avocent 24 port KVM w/ html5 interface is much more cost effective than 24 pikvms, etc.
My plan is to connect a single pi-kvm to the aging Avocent in my homelab rack. The Avocent can connect to dozens of devices, but the IP interface for it is old and unreliable. Hopefully the pi-KVM will make that experience painless.