Not from my experience. If you have too many keys and certain ssh agents like gnome keyring don't pick up the key intelligently and will try a all the keys in some order often resulting in the server giving rejecting you due to too many failures.
But you don't live with it -- you either move the extra keys to subdir, so that gnome keyring does not pick it; or use "IdentitiesOnly yes"/"IdentityFile foo" in .ssh/config to restrict certain hosts to certain keys (and yes, those work with ssh agent caching too).
I know many people just don't care about working tool, and tolerate the pain, but hopefully if someone knows enough to setup fail2ban, they should also be able to setup ssh config. Especially since reliable ssh connections is such a high quality of life improvement.
Yes I've hit this as well! Such an annoying behaviour.
However I think it's a good habit to make records in `~/.ssh/config` for each of your servers anyway just to keep tabs what, where, who, and with what keys.
Not from my experience. If you have too many keys and certain ssh agents like gnome keyring don't pick up the key intelligently and will try a all the keys in some order often resulting in the server giving rejecting you due to too many failures.