Things do indeed become much simpler when you reason in milliseconds since the epoch. So why is it so rare to do so? UNIX time is not milliseconds since the epoch. It's milliseconds since the epoch, plus the leap seconds that have accumulated over that period. The only widely used timebase that's pure milliseconds since the epoch that I'm aware of is GPS time, and basically everything adds in the leap seconds before actually using that number.
Ah, but negative numbers are not included in the set of counting numbers, and your initial comment referred to "the leap seconds" with no reference to "the leap second offset" or other verbiage that might imply anything other than a simple count of leap seconds.
Alas, I'm afraid you've no other option but to admit a minor error, as traumatic as that may be.
