It doesn't say only wealthy people will know what it is, it says they are much more likely to know what it is because they are much more likely to have experienced it first hand.

It would be hard to deny that generally sailboat racing is done by wealthy people.

That doesn’t follow. Poor people are aware of many things they don’t do. You’re wanting to lower standards to help people who don’t need that kind of help to expiate you own status guilt. Poor people may also not have cars. Or have ever flown on a plane or owned an iPhone. But they aren’t necessarily stupid and will probably know what these things are and if they happen to be then they should go to elite colleges.