You are correct that the key need not leave the HSM to transact, touché. However it is an essential property of valuable keys that they can be extracted for backup or replacement of the HSM, and often for availability. At least the various HSM systems I have worked with.

As I understand it, people have lost their wallet contents due to trusting email, Discord, DNS and SSL protected websites. So if there is no basis for trusting the other parties in an online transaction, it seems any action whatsoever could lead to financial ruin. Even moving my assets to cold storage makes the scenario that my heirs forget how multiparty sig recovery works or just some eager relative throwing away the box of USB drives away.