I would agree with you, but you seem to forget the concept of "cohorts" applied to this.

You could have people from age 20 to 40 working for everyone else; and people from 40 onwards to live on passive income. You would still have ~60% of the current workforce. Not impossible.