Your logic is not wrong, but I think the part you are omitting is that most managers really do not know how to manage. In most companies nobody REALLY has a good feel for how much work someone should get done in a given amount of time, or what fair compensation would actually be. This puts us into the mode of "pay the minimum amount we can, and then try to extract maximum work product".

If an employee is measurably unproductive for a few hours regularly, then the assumption is that their manager has failed somehow. I agree that this is most likely not an accurate conclusion, but it is a prevalent one.