A-GPS provides a current almanac, showing where all the satellites actually are. Without it, a cold start requires hunting for signals across a much wider range. As I understand it, older GPS receivers rely on finding a single satellite and waiting to acquire a full almanac from it while smartphones have enough compute to probably get a lock on multiple satellites without a complete dataset. The satellites will eventually broadcast the complete almanac and ephemerides, so a warm start shouldn't take as long.

D-GPS uses the known location and current readings from a nearby fixed receiver to increase the accuracy of your receiver, and does rely on knowing that you're in the vicinity, but it's not a feature of consumer phones.