We know neural networks cannot solve the halting problem. But isn’t the question whether they can learn the transition table for game of life? Since each cell depends only on neighbors, this is as easy as memorizing how each 3x3 tile transitions.
The halting problem doesn't mean you can never decide if something cycles etc, just that you can't always decide.
As it stands, my guess is that the LLM would always confidently make a decision, even if it were wrong, and then politely backtrack if you pushed backed, even if it were originally right.
Can you do a neural network that, given a starting position of the game of life, decides if it cycles or not? ;)
Ok, not cycles... dies, stabilizes, goes into a loop etc.