But for mastodon, even if it technically "doesn't matter" nobody stops there. You see things written like "The hardest part of getting started on Mastodon is the first step,"[1]
They talk about chosing a server based on region, language, topics of interest, or which ones your friends use. I've even some mention that your server choice can signal your political leanings.
I don't worry about any of those things when I choose an email provider or ISP.