Which is one reason why the Northeast Corridor works even though it's only sorta high-speed rail. There are a few cities off the coast like Pittsburgh that miss out but you handle a lot of population with a single line.

The only caveat is that, because it's not really high-speed, it works better as north and south segments than as one single line. But because it's NYC in the middle, even that works pretty well.