I have to assume there are some other obsessive engineers out there with reflections of their own. I'm curious how people approach the challenge of staying focused while keeping an open mind about the futility of a particular direction...
I mostly concoct and go through a whole idea over days. Then I let the dysfunctional aspect of me break it down the way it usually does which roughly goes like this, "okay, that's a pretty stupid idea, nay, worthless one". Then I just wait. Days, weeks. If the idea returns, then I tell the critical side of me to shut it and pursue the idea fully. These are for big ideas. It's scalable for smaller micro problems, but I never truly observed it there since I do the programmer thing of making trade-offs.
Something I've been trying for the past year or so is adopting the following mindset: "I'm going to stay on this direction until I prove to myself that it's going to work or I prove that it's not going to work." By "prove" here I mean semi-formal things like characterising the problem and the chosen direction well enough to say with high certainty whether or not it's going to work or is futile.
One upshot of this is that it forces me to be clear about the definitions of "going to work" and "not going to work." It's related to definition-of-done in some agile methods, and also to "imagine the problem as solved and work back from there," which is one of the heuristics Polya talks about in "How to Solve It".
I materialize my thoughts as I obsess over the problem in my notes. I find that doing this often helps me reframe the problem and the facts that I currently know, which helps me discard branches that will be fruitless.
