It's no more a distraction than opening a drink, taking a swig of said drink, or having to look around to see why the kids are crying in the back of the car.
Let's not suggest that Autopilot is safe completely unsupervised, and let's also not suggest that all drivers always look at the road 100% of every drive they take. There is a realistic middle ground, and I'm very grateful to have Autopilot to be an extra set of eyes.
