From what I understand, it is not the words themselves (thee etc) but the sounds that make the words - so the "th" and the "ee" are still legit sounds in modern English words. The network would just be synthesising the words you tell it to - it won't be picking the words for you.

I might be wrong though.