If you can retest everything, you can rebuild everything. But that doesn't help the network bandwidth issue. Debian doesn't have incremental binary diff packages anyway (delta debs) in the default repos anyway, so there's room for improvement there.
> If you can retest everything, you can rebuild everything
No, rebuilding is way heavier.
Even worse, some languages insist on building against fixed version of their dependencies.
It forces distributions to keep multiple version of the same library and this leads to a combinatorics explosion of packages to fix, backport, compile and test.
It's simply unsustainable and it's hurting distributions already.