I thought that was interesting too. I think perhaps it's something to do with the nature of the averaging process, where a bunch of faces averaged look better than the individual faces up to the point of being "good" looking, but exceptionally good looking faces have more to lose than gain.

For example, I've read that for women, the more contrast the better (which is why makeup works) and it's likely you lose a lot of that in the averaging process.