For some reason, our brains tend to overvalue savings. I've noticed this myself with gas prices. I am willing to go out of my way to stations that charge 10-20 cents less per gallon. Is that really worth the trouble? At ten gallons, this only amounts to 1-2 dollars out of 40 total. It really isn't worth it for me, yet I feel compelled to do it.
If it makes you feel any better, I guess you're performing a valuable service to society. You're keeping the market working the way it should by ensuring that stations with slightly lower prices get less business than stations with higher prices, which helps keep the prices down for the rest of us. So in addition to those one or two dollars of savings, you should also get the warm glowy feeling of being a public benefactor. Congratulations!
The easy way around this problem is to be aware of the value of your time. If it takes you another ten minutes to save 1-2 dollars and it feels like a chore, then it's obviously a bad deal.
Of course, your time isn't always worth the same. You may find yourself on a Sunday morning with time to burn (or retired/unemployed), in which case you might as well look for a better deal as a matter of principle. It also may be the case that going out of your way is enjoyable for other reasons and doesn't feel like a chore.
It's all about being aware of how the price you pay for anything is composed of many variables (time, money, sacrifices, etc) whose relative values are constantly readjusting.