Why is that? In nature, different fruits ripen at different times. Even if you were restricted to nature's cycle, there could potentially be fruit available for most of the year.

http://www.cuesa.org/page/seasonality-chart-fruit-and-nuts

I mean, sure, you wouldn't have access to all of these fruits in one location, but point being fruit wasn't something you got for a week in June, then never again until next June.