You may ramp up adult-level conversational ability faster, but it takes much longer to reach any state of native-like fluency.
For example, I lived and studied in Japan, and while I probably jumped from zero to ~middle-school in terms of basic communication ability in about a year with daily classwork and total immersion, I was nowhere near the average toddler in terms of basic fluency after that time. Even now, I can make myself understood in most adult-level contexts where a toddler would be helpless, but nobody would ever confuse me for a native toddler.
It's not just accent, either -- little kids still routinely teach me vocabulary and grammar.
If you had three years of paid vacation where you had to use the language exclusively for everything, your skills would probably improve to toddler-level or beyond. Though admittedly you already know many things that they have to learn at the same time.
Having to communicate with your family (and probably close friends/significant other/etc.) exclusively in the language probably makes a big difference as well.
I think a lot of fluency is having someone carry you around 24/7 and talk to you like a baby for a few years. Much of the rest is just having patient people around you for another 3-4 years who will listen to you babble like an idiot and correct you when you're wrong.
I agree that if I could do nothing but spend 3 years in total immersion, I'd probably accumulate an adult-level vocabulary (again, the ramp-up is steep), but I still wouldn't sound like a native at that point, whereas a toddler would sound like a native...but have a toddler vocabulary.
For example, I lived and studied in Japan, and while I probably jumped from zero to ~middle-school in terms of basic communication ability in about a year with daily classwork and total immersion, I was nowhere near the average toddler in terms of basic fluency after that time. Even now, I can make myself understood in most adult-level contexts where a toddler would be helpless, but nobody would ever confuse me for a native toddler.
It's not just accent, either -- little kids still routinely teach me vocabulary and grammar.