I find it interesting that reducing your EC2 usage by 50% decreased your expenses by 50%, it means one of two things, you aren't paying your employees and don't have any overhead, or the cost of EC2 dwarfs the cost of your employees and overhead.
If it's the latter, I'd seriously consider colo as you can probably reduce costs by another 80%.
Obviously the discussion was scoped around non-personnel expenses. I'm not going to dump an entire P&L here. And this is now wildly off-tangent.
I was illustrating that there is real world gain to be had by doing something as simple as switching to a new Ruby or spending some time with a profiler. These weren't drastic code rewrites. They didn't require layers of caching or sharding of my database. I fail to see what's even contentious about this.
As long as it can reasonably expected to be mostly bug free and support everything you need it to with little changes to the app. It wouldn't need to require to much time playing around with it before the EC2 savings are eaten up by the wage costs of spending time on it. (Depending on how many servers you are running of course)
This isn't a theoretical argument. I actually did this and it didn't take all that long and some of it was even fun. An added benefit is my specs run faster, too. So developer time is saved on every spec run now. You also hit that intangible of improved developer happiness.
Additionally, the X time exceeds Y cost argument really only works when people are optimally efficient. Clearly those of us posting HN comments have holes in our schedules that might be able to be filled with something else.
If it's the latter, I'd seriously consider colo as you can probably reduce costs by another 80%.