I guess it is about control in the end. Even if your employees are working completely remotely you could always check their output. But the problem is that typically a big part of middle managment has no idea how they would even start to do that. So the alternative is go there and talk to them while observing reactions — and derive your conclusions from that.

But that doesn't work as well remotely. You can't surprisingly be there and catch your employees at doing the wrong thing. What other unspeakable evils might they be doing while you are not watching?

Now of course you are right, just watching the numbers is not enough. But if your middle managment has no way of knowing if their subordinates are doing their jobs remotely, your business would be in a blind spot even if not a single person worked remotely.