I'm not saying the system is perfect, but there is a reason we switched to the current system.
If I've invented widgets (or think that I've invented widgets), I should be able to know if I can patent and sell them, without years later getting sued because someone once did it in their basement and left it at that.
First to invent doesn't automatically mean they can sue when you file your own patent and intend to sell - they have to file their own patent first, and if you've already published enough of the invention before they even filed, then that only means neither side gets a patent (original inventor because they can't wait until after publication to file, second inventor because they were late).
Also, first to file has the same problem you mention. You can invent something, never publish, and get sued by somebody who reinvented it later.
If I've invented widgets (or think that I've invented widgets), I should be able to know if I can patent and sell them, without years later getting sued because someone once did it in their basement and left it at that.