It's a pain point with the entire design of federated services. On the other hand, the pain point with the monolithic services design of Facebook is I can't even talk to people on other services so...

I'd rather be able to email everybody and have it be annoying to switch than be able to only email people on my chosen provider (and then have to make an account on every service anyway).