The most accurate way to phrase it would be to say that "natural selection is about less fit individuals having a decreased chance of passing on their genes."

Evolution is really about which genes are passed on to the next generation. Most of the population at any given time is going to be less-fit than the fittest (by definition). That doesn't mean they don't pass their genes on. While totally unfit people may not be able to pass on their genes at all, many, if not most, of a generally less-fit population will pass on their genes at least one generation. Depending on the quality of the mate they are able to obtain, their genes may not go further than the one generation, or they may keep going for many generations. On average, the less-fit genes will "die off" but this isn't always the case, for many reasons.