But code that has never been executed can have orders of magnitude more categories of bug than code that works at least once. And I'm good enough that 99% of the time I don't miss the corner cases -- it's the more blatant problems that tend to bite me, and those are eliminated by making sure all the code gets executed at least once. Once my code is wired up right, it tends to work. But I won't know if it's wired up right until it's been executed once.
>Lack of complete test environments for your mandatory 3rd party APIs suck!
Try writing code for systems without the ability to debug. Or where something is going wrong and there's no way to figure out why based on logging or debugging. Or without complete (or correct!) documentation. Or for projects where, once you ship, there is no option to patch your app, so it needs to be as close as possible to bug free. Thankfully modern platforms (mostly!) avoid the last pitfall, but I've been there, along with the other situations. And I win these battles.
>I suspect we have a fundamental disagreement of opinion as to what constitutes "overkill".
No, I'm just not communicating effectively. And it's no longer worth it for me to try any harder.
FWIW, I'm not talking about complete test coverage. The apps I usually write (games) are almost entirely pointless to write tests for. Ever. Sure you can for some specific libraries you're using, but for the actual game itself? Hardly ever worth it.
I've shipped games with zero chance for patching, some pretty popular -- and I didn't hear of any bugs worth mentioning after release. And there were no tests involved in the development process (other than the kind where QA tests the game and tells me what doesn't work). I know that concept is a bit alien on HN, but it's rare to find a game (or even a game engine) with a nontrivial test suite.
Most game bugs wouldn't be prevented with unit tests anyway -- probably only one in a hundred at best. They almost always result from emergent complexity, not from a single function returning a bad value.
> it's the more blatant problems that tend to bite me, and those are eliminated by making sure all the code gets executed at least once
Maybe I'm taking your repeated reference to testing all 256 permutations too literally.
> Try writing code for systems without the ability to debug. Or where something is going wrong and there's no way to figure out why based on logging or debugging. Or without complete (or correct!) documentation. Or for projects where, once you ship, there is no option to patch your app, so it needs to be as close as possible to bug free. Thankfully modern platforms (mostly!) avoid the last pitfall, but I've been there, along with the other situations. And I win these battles.
At least I've dodged one of those bullets: I haven't worked on anything outright unpatchable just yet :).
> No, I'm just not communicating effectively. And it's no longer worth it for me to try any harder.
Of course it was. It's always about bugs.
But code that has never been executed can have orders of magnitude more categories of bug than code that works at least once. And I'm good enough that 99% of the time I don't miss the corner cases -- it's the more blatant problems that tend to bite me, and those are eliminated by making sure all the code gets executed at least once. Once my code is wired up right, it tends to work. But I won't know if it's wired up right until it's been executed once.
>Lack of complete test environments for your mandatory 3rd party APIs suck!
Try writing code for systems without the ability to debug. Or where something is going wrong and there's no way to figure out why based on logging or debugging. Or without complete (or correct!) documentation. Or for projects where, once you ship, there is no option to patch your app, so it needs to be as close as possible to bug free. Thankfully modern platforms (mostly!) avoid the last pitfall, but I've been there, along with the other situations. And I win these battles.
>I suspect we have a fundamental disagreement of opinion as to what constitutes "overkill".
No, I'm just not communicating effectively. And it's no longer worth it for me to try any harder.
FWIW, I'm not talking about complete test coverage. The apps I usually write (games) are almost entirely pointless to write tests for. Ever. Sure you can for some specific libraries you're using, but for the actual game itself? Hardly ever worth it.
I've shipped games with zero chance for patching, some pretty popular -- and I didn't hear of any bugs worth mentioning after release. And there were no tests involved in the development process (other than the kind where QA tests the game and tells me what doesn't work). I know that concept is a bit alien on HN, but it's rare to find a game (or even a game engine) with a nontrivial test suite.
Most game bugs wouldn't be prevented with unit tests anyway -- probably only one in a hundred at best. They almost always result from emergent complexity, not from a single function returning a bad value.