I'm always suspicious of things "planned to survive thousands of years". I'll bet politicians in Rome and Sparta big-noted themselves with "projects which'll last thousands of years!" (which they then contracted out to their brother-in-law). So far as I can tell there's not much other than Pyramids which humanity has designed/built that could plausibly claim to have "survived thousands of years", and even those didn't come close to being impregnable enough to be considered "safe" for keeping curious humans away from long half life radioactive trash...
Agreed. And Hammam Essalihine/Aquae Flavianae is a Roman bath which is still in use.
Regarding the Pont du Gard, it's intact "due to the importance of its secondary function, as a toll bridge. For centuries the local lords and bishops were responsible for its upkeep, in exchange for the right to levy tolls on travellers using it to cross the river."
I believe bigiain's point is that the structures needed active upkeep to last thousands of years.
Your point is true about people robbing the masonry, but that's closely related to bigiain's comment about "keeping curious humans away from long half life radioactive trash".
I should add that I believe the plan is to do significant work in sealing compartments so anyone looking to wreak havoc would need to breach the facility, get down a significant distance and then work through a huge amount of concrete or clay or whatever it would be.
I think the supply line from port to the facility would be a softer target and is something the jury and stakeholders would be considering in serious detail.
