As I understand things, in Japanese the terms "onii-san", "ojisan" and "ojii-san" literally translate as "older brother", "uncle" and "grandpa" respectively - and are widely used for people who aren't relatives.
So while the colloquial use of ojisan roughly lines up with "middle-aged man" it's not a perfect mapping.
Ojisan age bracket can be pretty wide, but 81 sounds to me more like an ojiichan. That said, if they're running a business and are still somewhat lively they can possibly still pass as ojisan in context.
Oniisan is much younger usually; usually I don't hear ojiisan so much as ojiichan (as well as ojisan rather than ojichan), my wife is Japanese and it rubs her weird to hear the opposite honorific in these cases.
“Middle-aged” is a wrong in this case translation of “ojisan”, lit “uncle” but colloquially can mean “old man” in an endearing or mean way depending on context
Well, if the edges are the Japanese life expectancy of 84 and 0, then 80 does fall somewhere in the middle. So there should be some four-year-olds in this deck, too.