But you don't live with it -- you either move the extra keys to subdir, so that gnome keyring does not pick it; or use "IdentitiesOnly yes"/"IdentityFile foo" in .ssh/config to restrict certain hosts to certain keys (and yes, those work with ssh agent caching too).
I know many people just don't care about working tool, and tolerate the pain, but hopefully if someone knows enough to setup fail2ban, they should also be able to setup ssh config. Especially since reliable ssh connections is such a high quality of life improvement.
But you don't live with it -- you either move the extra keys to subdir, so that gnome keyring does not pick it; or use "IdentitiesOnly yes"/"IdentityFile foo" in .ssh/config to restrict certain hosts to certain keys (and yes, those work with ssh agent caching too).
I know many people just don't care about working tool, and tolerate the pain, but hopefully if someone knows enough to setup fail2ban, they should also be able to setup ssh config. Especially since reliable ssh connections is such a high quality of life improvement.