A momentary distraction like a quick HN check is not the same as the context switching of, for example, going to a budget meeting while in the middle of trying to figure out some technical detail. Very different things.
The difference is scheduling and autonomy. If you had "HN Time" blocked out in your calendar it'd be just as distracting as a budgeting meeting. HN only is different because you can choose when to be distracted by it, and you're happy to say no to it.
If you could control when the budget meeting happens, or if you could decline it, you'd have far fewer issues with it.