Every time I spin up a box by hand (which isn't often), I'll set up a sudo user with my SSH key, then drop out of the root shell and log back in as the new user. Only then will I disable root login and password auth over SSH and start setting up fail2ban and the like.
If I lose access by, say, switching to a new computer and losing my SSH key, then I'm more or less dead in the water. But it's a small price to pay and 1password supports SSH keys first-class now.
at that point, why not just do FIDO2 with a YubiKey or something? eg https://www.ajfriesen.com/yubikey-ssh-key/ as long as you don't lose the YubiKey or the backup YubiKeys you're good.
I use it with ssh and password auth disabled, is there a reason not to? Might be overkill but the host is in my home so physical access if I ever get locked out is not an issue.
I think this is still reasonable, attackers may have a database of leaked keys (e.g. if you ever accidentally commited to GitHub, or ever ran a malicious script which uploaded it), which they then try on random servers.