That's my experience too (using gpg for ssh). While I remember that gpg agent can die once in a while it's very rare and the benefit of having keys in a separate device and using them on any machine instantaneously is certainly worth it. Not to mention U2F and built in TOTP. I can easily login in on a friend's machine without sharing any private/secret keys.