Yes. "What is the most probable next word given the text so far" is the standard 'language modeling' task from classical NLP where it was done before using Markov chains and n-grams. RNNs then transformers and huge amounts of training data made them output what we see now, versus coherent but otherwise not very impressive text 10 years ago. The large contexts of current models made it possible to generate valid code where you need to remember to close a bracket opened 200 tokens before.