There's usually some regression to the genetic mean for children from people who are extreme outliers. Even if you have "smart" genes, you're unlikely to luck out on such an outstanding arrangement of genetic material again... and von Neumann was a very extreme outlier. By some accounts, the most intelligent human in history [0].
You'd expect for any of his kids to be outstanding, but not earth-shatteringly so. And looks like that's about right if you look up his daughter [1].
It doesn't help that the aristocratic educational system of having a handful of tutor/mentors as the primary means of education has fallen (steeply) out of fashion.
People greatly underestimate just how effective children can learn. They're basically capable of absorbing as much information as you can pump into them. It's a delicate balance making sure they learn enough basic human operating stuff in addition to everything else, but the results can be remarkable.
I remember hanging out at camp with a kid who loved board games. Especially probability based ones, such liar's dice and Dominion, but also more general hidden information games like Stratego. Kid was undeniably a prodigy, and he kicked my ass every time. I had just graduated university with studying CS and Physics, this kid was looking forward to starting kindergarten in the fall.
