I've had this a few times too. I assume it's a driver trick to drive in the wrong direction so the rider has to cancel, pay the cancellation fee, and you don't even get to rate the driver in that case so it doesn't affect their score.

I'm not sure why they don't just decline the fare, but maybe they don't see the destination until they accept and they don't want to go there, or maybe Uber penalizes them for declining too many fares and this is a hack around that.