You are shifting goals, from the population growth to the fertility rate. Indeed some countries more that others are growing because of the migration and not fertility. Also that's not a new phenomena at all.
All that doesn't negate the fact of the population growth.
As a consequence of that growth we need more housing, and that's not happening in a needed rate.