Though it doesn’t mention it by name exactly, I think a related idea for systems that are optimized close to a point of phase change is “the edge of chaos”

https://en.m.wikipedia.org/wiki/Edge_of_chaos