Investors who do not rent out locally - including those renting out to tourists, those keeping empty homes for occasional vacation use, those keeping empty homes because they're playing financial shenanigans (moving money offshore, money laundering, keeping some properties empty to prevent depressing rents from nearby properties). Probably some other groups I'm not even aware of.
Point being, it's not true that all supply of homes ends up being used by people to live in. A lot stay empty most, or all, of the time. People owning those tend to have more money than locals looking for a place to live, so the latter get priced out.