Modern US houses, at least in much of the country, aren't built to be livable or to survive themselves without year-round temperature control. Partly, this is because optimizing for building survival and inhabitant comfort without AC and with only limited heating is at-odds with designing for efficient whole-house heating and cooling.
Notably, you don't want a very tight, low-outdoor-air-exchange envelope and tons of insulation in your walls if you're going AC-free and not also trying to start a mold farm. And if you want efficient AC you don't want tons of huge, openable windows on every single wall, at the end of every hallway, in all rooms, et c, nor an open, breezy attic (which helps a ton with temp control if you don't have AC, but is just wasteful if you've got it)
AFAIK design for minimizing AC power use and design for being able to tolerate not having AC are a "pick one, and only one" kind of thing.
[EDIT] Oh, and not strictly related to AC efficiency, but it takes a lot more skill and care in material selection to build a house that won't have all kinds of problems after just a few years of natural outdoor temperature swings, in the molding, flooring, even wall alignment & finishes (drywall, mostly, these days), so of course no-one bothers anymore—let the interior hit 90s in the Summer, swinging a bunch day-to-night, with high humidity, and parts that aren't near your heating source reach below-freezing temps at night in the Winter, and your interior will age at least 5 years for every 1 actual year, thanks to damage from huge amounts of expansion and contraction, unless it's designed to account for that. Ditto with aligning houses for optimal airflow, as far as the direction they face. Why bother, since it's always gonna have whole-house AC and heating? So no-one does.
Notably, you don't want a very tight, low-outdoor-air-exchange envelope and tons of insulation in your walls if you're going AC-free and not also trying to start a mold farm. And if you want efficient AC you don't want tons of huge, openable windows on every single wall, at the end of every hallway, in all rooms, et c, nor an open, breezy attic (which helps a ton with temp control if you don't have AC, but is just wasteful if you've got it)
AFAIK design for minimizing AC power use and design for being able to tolerate not having AC are a "pick one, and only one" kind of thing.
[EDIT] Oh, and not strictly related to AC efficiency, but it takes a lot more skill and care in material selection to build a house that won't have all kinds of problems after just a few years of natural outdoor temperature swings, in the molding, flooring, even wall alignment & finishes (drywall, mostly, these days), so of course no-one bothers anymore—let the interior hit 90s in the Summer, swinging a bunch day-to-night, with high humidity, and parts that aren't near your heating source reach below-freezing temps at night in the Winter, and your interior will age at least 5 years for every 1 actual year, thanks to damage from huge amounts of expansion and contraction, unless it's designed to account for that. Ditto with aligning houses for optimal airflow, as far as the direction they face. Why bother, since it's always gonna have whole-house AC and heating? So no-one does.