My completely amateur understanding of chip manufacturing progress is that it is not bound by routing problems or similar (which I am guessing you are referring to), but by lithography physics, chemistry, etc. I might be wrong though, are you saying that faster chips are actually becoming possible because more compute power has become available through the previous generation of chips, at this point?
The lithography has obviously been much more salient, and still is, but both are limiting factors.
To draw a parallel: as the improvement of computing hardware performance slows down, one would expect more attention to be paid to software performance optimization, so that more of the potential of the hardware is actually used.
In the same vein, as lithography improvements slow down, one would expect more attention to be paid to e.g. the routing problem (but also placement of gates and buffers/repeaters, logic optimizations for timing closure, and others) so that more of the potential of the lithography is actually used.
So:
> are you saying that faster chips are actually becoming possible because more compute power has become available through the previous generation of chips, at this point?
Yes.
This is obvious when you look at it in the extreme: chips like the Intel 8080 were designed by hand. As in, the shapes to be created via lithography were drawn by hand. Doing this for a modern CPU core is plain impossible. We can only design those because older CPU cores already exist and can run automated tools that do this job for us.
It's less obvious but still true for generational improvements. The chip design problems are becoming computationally harder in each generation, for two main reasons. First, the designs are simply becoming bigger (more gates, more wires). Second, the design rules are becoming more complex.[0] If you kept the chips that you use to create the design constant, then each design generation would take more wall-time than the previous one to go through the automated design tools, until at some point everything just becomes infeasibly slow.
Using faster chips to run the physical design tools on allows the designs to be processed faster, which leads to a faster feedback loop with logic designers (the people who write Verilog), which then leads to a faster feedback loop with architects, which ultimately leads to faster/better designs for the next generation of chips. This may be surprising for software developers -- we get grumpy when our feedback loops take minutes. Chip designers have to deal with feedback loops that take days in some cases for the physical design.
[0] For example, at the lowest routing layers you can't just draw wires wherever you like, but have to satisfy some pretty byzantine rules about things like minimum metal area, distance to neighboring wires, allowed shapes of wires, allowed shapes of holes between wires, and so on. These rules exist because of limitations in the capabilities of the lithography: the process engineers are able to make features smaller, but at the cost of forbidding certain shapes that would cause problems for physical or chemical reasons.
