I think 'chaotic' is the technical term. It is technically deterministic, but the system is incomprehensibly complex and sensitive, such that it may as well be non deterministic / pseudorandom.

You could drive the same car down the same road at the same time tomorrow, and it may behave differently. It makes bugs / edges cases impossible to reproduce and fix.

With a human you can just say "why did you do that? Don't do that." Little fine tuned tweaks to neural nets is not so easy.

> How is it non-deterministic?

It's non-deterministic in a practical sense; you cannot reproduce the exact circumstances that caused a certain behaviour. Maybe a bug flew by one of the cameras at a certain moment, or a pedestrian with certain type of clothing passed by. You get the idea.

Depends. In principle you can log exactly what data the camera sends to the decision making unit. And you could replay that data.

The camera itself might not be deterministic enough to reproduce the same data with the same beetle, but replaying the log should work.

(Keep the log in a 'black box' on the car, and perhaps only keep the last x hours of data, if necessary.)

I think Waymo already does something like these kinds of replays.

what you’re referring to is just simulating, at least after the point your modified algorithm does something slightly different in the simulator than the real car did in real life.

I wondered about the last point, too. But Waymo apparently worked out a satisfactory solution to that.