Most RTCs do use the same crystals that are in wristwatches, which is actually the source of their error.
The tuning fork crystal design used in wrist watches has a parabolic temperature coefficient, which means that the clock is only really accurate at room temperature. This isn't a problem for wristwatches, because, presumably your wrist is at approximately room temperature, but it does become a problem for electronics that operate with large temperature swings. (like the inside of a phone or computer, for instance).
The vast majority of quartz references in consumer devices could be a lot better, but they are not fine-tuned at the factory. They're just "close enough". So they drift a lot.
A simple tuning procedure could make almost all quartz clocks you own a heck of a lot better. But in many cases this is hard to do because the manufacturer never put the tuning components (a variable capacitor or the like) on the PCB.
The tuning fork crystal design used in wrist watches has a parabolic temperature coefficient, which means that the clock is only really accurate at room temperature. This isn't a problem for wristwatches, because, presumably your wrist is at approximately room temperature, but it does become a problem for electronics that operate with large temperature swings. (like the inside of a phone or computer, for instance).
https://www.maximintegrated.com/en/app-notes/index.mvp/id/58