I think this discussion is 20 years too late, the only reasonable way forward is IPv6 or bust. We're reached the end of the IPv4 space and the pressure is mounting; any imaginable backwards-compatible technology would have to run for many years in a limited mode where it's basically just a better way to do NAT traversal and only then, when most endpoints are compatible, you would get to see the address-space relief benefits.
Another nail in the coffin of a graceful upgrade from IPv4 is the widespread filtering of IP options in the backbone, the only practical way I know you could craft an extended IPv4 packet that is still routable by the legacy infrastructure while forwarding and maintaining all the extra header data required for routing in the IPng realms. This was not necessarily true in the early 90s, and many hardware generations would have had the ability to fix it.
So I somewhat disagree with the GP that IPv4 was not forward compatible: it included a mechanism for just that in the form of IP options that seemed like a good idea in 1983, but which proved technically inappropriate for the future needs of the internet. So you can't really fault IETF for wanting to break away from that and earnestly thinking people would just upgrade.
Another nail in the coffin of a graceful upgrade from IPv4 is the widespread filtering of IP options in the backbone, the only practical way I know you could craft an extended IPv4 packet that is still routable by the legacy infrastructure while forwarding and maintaining all the extra header data required for routing in the IPng realms. This was not necessarily true in the early 90s, and many hardware generations would have had the ability to fix it.
So I somewhat disagree with the GP that IPv4 was not forward compatible: it included a mechanism for just that in the form of IP options that seemed like a good idea in 1983, but which proved technically inappropriate for the future needs of the internet. So you can't really fault IETF for wanting to break away from that and earnestly thinking people would just upgrade.