Overfitting is lack of genralisation, LLMs can produce novel output by using abstraction composition, which you cannot do without generalisation.

Other signs of non-overfitting: Abstraction laddering, task decomposition, novel (i.e. unseen) joke explanation