Yeah that was a little bit optimistic about the powers of Agile for me, but I think the thing that Agile gets right is that we all have to operate under the uncertainty of the scope of software projects. In that sense it's not a methodology for estimating the complexity of a software project but a way to start working without that.

Perhaps the biggest reason this keeps causing problems is that companies have no good way of dealing with change. If you expect to finish any project that you start then you need to know more about that project than is realistic at the time you start it. A canceled project is a big failure for most employees and not something they want to happen to their careers.

It's not so much avoiding estimating complexity as much as moving from absolute estimates to relative estimates. Arlo Belshee has the best article I've seen on the subject: http://arlobelshee.com/post/planning-with-any-hope-of-accura...

However, until you establish reality, all the estimates in the world aren't going to help much. And most customers cannot estimate reality until they are actually in the process.

I'm not saying Agile is a silver bullet -- it can go wrong in many ways, and it's not appropriate for every situatoin. However, it's the best we have for its niche.