Exactly. Billionaires are truly international citizens, shifting their wealth to the most amenable climates (often literally, e.g. Florida), so few of them will be truly invested enough in a certain locale for cost of living to matter.

And if they are, well then they own the entire town, so they definitely don't care.