Hacker News new | past | comments | ask | show | jobs | submit login
Leap second hiatus (fanf.dreamwidth.org)
183 points by ckarmann on Nov 19, 2020 | hide | past | favorite | 113 comments



Can we take a longer hiatus on leap seconds, maybe 79 years or so, and only update once a century? Local apparent noon being off by 30 seconds or so has approximately zero impact on my daily life, but stupid things in datetime libraries do occasionally have impact. If we can’t throw off the oppression of UTC and greet TAI as liberators, at least adjust the clock at regular, very infrequent dates.


Besides the sibling comment that some things rely on the days being synced to rotation, doing it once a century would pretty much guarantee that only the most obsessively robust systems would take it into consideration when being built.

We'd be giving ourselves a Y2k-equivalent to deal with every century. Maybe that's not the biggest deal in the world, but I still remember the last viscerally and we're already a fifth of the way to the next one (not to mention the timestamp bug one coming up in a decade).

I disagree with the other comment's suggestion that leap hours can be taken into consideration like DST. I see no reason why DST code (which is generally implemented as an extra timezone) could straightforwardly be adapted to handle a single permanent shift of time.


"If it hurts, do it more often" — Martin Fowler


Practice makes perfect. Release early, release often.


> I see no reason why DST code (which is generally implemented as an extra timezone) could straightforwardly be adapted to handle a single permanent shift of time.

The point is that we don't have to add the additional code to handle leap hours, as it can be implemented as a large swath of time zone changes.

We can imagine that there would be two versions of UTC, one before the leap hour (hereafter UTC) and one after the leap hour (hereafter UTC'). After the leap hour UTC and UTC' will coexist, where UTC' = UTC-1. Since time zone definitions themselves have UTC offsets, the trick is that instead of replacing all occurrences of UTC with UTC' (so that they would have UTC' offsets) we replace those offsets to have the same net effect. For example EST is UTC-5 before the leap hour and UTC'-5 = UTC-6 after the leap hour, so we pretend that the definition of EST changed to UTC-6.

Note that this approach essentially fixes the UTC-TAI offset (as opposed to the UTC'-TAI offset) and makes UNIX timestamp (time_t) deviate from the current version of UTC (well, UTC'). I believe this is just a matter of definition and thus much doable than changing UTC every few year. After all, leap hours would be much rare, probably the first one would occur at least a few centuries later.


The problem is not that changing the code is complicated. Changing code to accommodate four-digit years wasn't complicated. The problem is that you have to change every single piece of software which uses time. What you're proposing is exactly what I think needs to be avoided.


You would be right if vendors weren't able to deliver time zones in time. In practice they are, though with lots of headaches. We can just piggyback on the existing infrastructure of the time zone distribution with no additional cost.


I don't think this infrastructure you're relying on exists. A great deal of software isn't maintained at all. It was built once by a contractor and has been plodding along doing its work for decades.

Major software vendors won't have any particular trouble, but neither did they have any trouble in 2000. It's all the tail-end stuff which is in danger of breaking.


It should be very visible to users that softwares are unable to keep with time zone changes anyway (including those due to the relocation). If they are able to keep up somehow, probably manually, then leap hours don't impose any additional requirement to them.

Or, you can abolish the time zone to make software developers very happy [1]. At the expense of everything else.

[1] https://qntm.org/abolish


If you don't do any tzdata updates in a century then I can guarantee you that the application will be showing wrong local times anyways. Timezone changes happen. That is a already occurring thing.

And it's not like there is even hard deadline to do the changes, nor would it come unpredictably. If any schedule we set would seem problematic, the change can be deferred by another decade or two to get systems fixed if really needed.


> doing it once a century would pretty much guarantee that only the most obsessively robust systems would take it into consideration when being built.

tzdata updates happen all the time (in a relative sense). So code would need to handle those, and I don't see why potential leap hours couldn't be handled like that. Just means that Europe/London eventually will be +01:00 (or is it -01:00, anyways) instead of +00:00 its now.


Yes, I think that would make sense, but leap hours would be very rare: the first one won’t be needed for 500 years. See the first table at https://www.ucolick.org/~sla/leapsecs/dutc.html

But, leap hours don’t need to be co-ordinated: countries can and do change their clocks to adjust the time of sunrise when they want.

And leap hours will work much longer than leap seconds: leap seconds will become too frequent in 1 or 2 thousand years, but leap hours will just be getting started. See the second table at the same link https://www.ucolick.org/~sla/leapsecs/dutc.html


As a former developer of software for an international shipping company: Yup. They happen a lot more frequently than most people realise, I expect. Political reasons (e.g. neighbouring states [as in Sovereign] deciding to unify timezones), coups/wars suspending DST, and so on all usually result in some kind of adjustment, even if it's just the name of the timezone changing.


I can't speak to how different parts of society will interact in hundreds of years, but if today there were going to be a leap hour and programmers requested that we start considering London to be in +01:00, the chances of politicians or public agreeing with that request is nil.


Sure, we will randomly move the end of the DST from the last Sunday of October to the first Sunday of November and everyone will try to kill us, right? DST is the perfect example that we have a relative freedom in changing time zone offsets.

You are however right that this is a political matter requiring some (but I believe, reasonable) amount of coordination, but timekeeping is in many parts political anyway. Practically speaking countries would implement the change out of necessity and not because of agreements, as one hour deviation is significant enough.


> we have a relative freedom in changing time zone offsets

Who is we in this sentence? Programmers certainly don't, they have to report times and timezones in their software as the populace expects it to be reported. Governments can change timezones, but they're not going to, because the political cost of telling everyone that they're in a new timezone is greater than the political cost of telling programmers where to shove it.

Most of the public will see absolutely no reason why they should have to permanently change the timezone they're in for the sake of a one-time event. And honestly, even as a programmer who'd be doing the fixing, I agree that they're right.


Governments absolutely can. The updates to tzdata [1] are full of arbitrary changes happening all around the globe, not just for a few weird countries. The example I quoted above is 2007 changes to the United State's DST and a perfect example that governments can push time zone changes with at most marginal perceived benefits. Also remember that the current proposal to abolish leap seconds itself was primarily arosen due to computing complications anyway. Programmers can (indirectly) affect the future of leap seconds than probably any other group else.

[1] https://github.com/eggert/tz/blob/master/NEWS


Russia decided to just stay on DST with 6 weeks notice a few years ago, thereby instantly creating half a dozen new timezones. And now they're discussing going back to normal time/DST...


I don't see a single thing different between leap hour and DST switch. Especially since Europe wants to cancel DST it wont even be something which happens periodically forever


> Besides the sibling comment that some things rely on the days being synced to rotation

Examples would be great. Especially examples of things relying civil time to track rotation.


People who can't cope with leap seconds should stop pestering the people who define UTC and just switch to TAI. They can do that now. No one is stopping them. Leap seconds are a pain and I myself would happily choose TAI over UTC for the system clock on a typical embedded system.

However, for civil time keeping and for some technical purposes, UTC as currently defined is exactly what we need. Don't fuck with it.


No, they can't.

Programmers have to deal with the outside world. The outside world has timezones defined as offsets from UTC. So when UTC changes, the whole world changes their clocks, and programmers have to cope. If your program doesn't agree with that world about what time it is, your program will get blamed.

If it was a choice that I could individually make, I would tell astronomers to go pound sand. But it isn't. By historical happenstance, they have been put in charge of time, and it is truly a question of the tail wagging the dog when they care about a 1 second deviation between UTC and the position of the sun at a particular longitude.


Why not derive UTC from TAI (via a table of leap seconds)? Then apply the TZ rules to DST to arrive at local time.


The short answer is that I don't do this because I'm not insane.

Think about it. How much software would I have to rewrite to make this possible? What value does that rewrite create? And will maintenance programmers thank me for it? Will it get adopted?

As a programmer I have to interact with databases, operating systems, various libraries in various languages, and so on. Frequently with quick switching between them. ALL are written assuming UTC. Shall I name myself Don Quixote and spend the rest of my life on a rewrite that nobody wants to us?

Just to give a sense, I recently had to do some data science tasks. Data was in Postgres, my code was in Python, using pandas for speed, and pandas is built on numpy. All four of those have their own datetime library, and I had to deal with all four of them. If I followed your suggestion, before I did any work I'd have to rewrite all 4 libraries, and then figure out how to replace Amazon RDS because it doesn't support my "improved Postgres".

And if I did all of that, I've not changed the odds of something like https://www.wired.com/2012/07/leap-second-glitch-explained/ happening again and taking down my favorite websites for a while. Because guess what, they won't be running my super-duper rewritten code.


I feel the solution is for the programming community to slowly and methodically move towards using TAI in basic infrastructure (operating systems, routers, NTP etc) and build a base on which to derive the civil timekeeping (UTC + localtime, DST etc).

That, or get involved in the standards work. That said, after ruining observations by sending a bunch of satellites into space, tech people might be even less popular than before in the astro community...

Edit I just got around to reading that Wired article and... ugh

> Reddit was just one of several web outfits that were hit by leap second glitches just after midnight Greenwich Mean Time on Saturday,

Article is from Jul 2012, GMT was not in effect at that time. I guess they meant UTC, but if tech journalists don't know that GMT is depreciated as a concept of "universal time" what hope is there for meaningful time reform...


Do you have any idea how much work it is to rewrite all of that software?

The solution people are actually moving to is https://developers.google.com/time/smear. The downside is that measurements of elapsed time will be off in that day. But no rewrite needed, and most applications are going to be just fine with that.

Article is from Jul 2012, GMT was not in effect at that time. I guess they meant UTC, but if tech journalists don't know that GMT is depreciated as a concept of "universal time" what hope is there for meaningful time reform...

Timezones may have multiple names. Greenwich Mean Time and Zulu, aka GMT and Z, are officially defined timezones that happen to be UTC+0. Both are in widespread use and are not wrong.


There’s an ambiguity in that GMT is a civil timezone that in the UK is BST when DST is in effect. UTC cannot be affected by DST.

I hear a lot of Brits calling their time “GMT” even when DST is in effect. It’s a shorthand for “current time in the UK”. Hence, ambiguous.


> However, for civil time keeping and for some technical purposes, UTC as currently defined is exactly what we need. Don't fuck with it.

You are asserting that without any justification. I see no benefits and lots of downsides in having leap seconds in civil time specifically.


I think that the point is that for most definitions of "civil timekeeping" you just do what Unix does and ignore the leap seconds altogether and simply live with the fact that once in a while one wall-clock second takes two SI seconds (or does not exist at all), probably by simply being completely oblivious to that fact.


Don't even get me started on UNIX time. On that I agree with the parent comment, whatever the time scale, you shouldn't fuck with it. Alas, UNIX is what it is..


I'm a diehard proponent of leap hours [1]. We already have a regular (or, rather irregular) adjustment in the civil time, namely time zones and "daylight saving" times. While I passionately hate DST, if we have to have DST anyway it's much better to make UTC adjustments use the same increment as well.

[1] http://www.leapsecond.com/LEAPHOUR/


> If we can’t throw off the oppression of UTC and greet TAI as liberators, at least adjust the clock at regular, very infrequent dates.

Why not just use TAI for almost everything and leave UTC (including regular leap seconds) for applications that need it?


The more infrequent your deployments, the more difficult they will be.


How does TAI relate to Epoch time?

More and more I use Epoch time for most things just because of how convenient it is.

I rarely need sub-second granularity, and it's just one number, so it's sortable, math-able, etc.

Then whoever is doing the display can convert it any way they want to.


Remember that your code needs to account for the possibility of a negative leap second, meaning if you're using epoch time for timestamps you could have an instance that an event that happened after a previous one could be stamped as having happened before it instead.

https://en.wikipedia.org/wiki/Leap_second

Edit: Actually I was slightly wrong, in the event of a leap second, we would repeat the same second twice.


Thank you for the heads-up!

I try to write code loose enough that this would not be a problem. :)


Can we take this solved problem and replace it with a new one that has details we haven't figured out?


Pretty sure there are lots of things that you use everyday that actually require precision time, such as GPS.



Leap seconds have nothing to do with precision time. It's a non-periodic alteration to the definition of UTC. Those niche applications that require second-accurate knowledge of Earth's rotational accuracy don't need to rely on leap second changes to UTC; they could get that information out of band, and will need to anyway when they need sub-second accuracy. The 99.999% of other applications that don't need to know Earth's rotation orientation relative to the sun within a second -- but do have to calculate time differences between two UTC times -- would be much better off not having to deal with leap seconds. Leap seconds were a terribly misguided idea.


> Leap seconds were a terribly misguided idea.

I prefer dropping leap seconds, but I wouldn't call them misguided. UTC and leap seconds come from maritime celestial navigation, where tracking rotation is actually important. Civil time then just piggybacked on that, which at the time probably was perfectly reasonable solution.


A detailed look at the negotiations that led to leap seconds shows that they were not for maritime celestial navigation. During the process several different times the celestial navigation folks set a limit on how far radio broadcast time signals could deviate from astronomical time, and every one of those limits was violated as the negotiations proceeded. By the time the draft recommendation was given to 12th Plenary Assembly of the CCIR it allowed for leaps of multiple seconds, and that draft was amended on the floor to leaps of only one second before they voted to approve.


Precision isn't a binary, it's a continuum. The question at hand is whether there are significant use cases where 1 second of slack in matching the Earth's rotation is acceptable, but 30 seconds is too much.

GPS isn't an example of that because 1 second is already a quarter mile off.


Pretty much all telescopes (under computer/time control) on this planet rely on UTC being quite close to UT1, since every second difference leads to a 15 arcsecond error in the pointing of the telescope (error between where you want to point the telescope on the sky and where you are actually pointing).

While <5s is usually not a problem (except for some instruments with very very small FoV), at 30s it really becomes a problem for instruments with modest fields of view.


I would think this is an argument for telescopes using UT1, not whether UTC should be adjusted for leap seconds. This is such a fringe application and all involved are aware of the issue, that I don't think it's relevant here at all.


UTC is defined to be within 1s of mean solar time at 0deg longitude (= UT1).

If leap seconds are a problem for the intended application then UTC is the wrong thing to use. There are other scales that are defined differently that can be used in it's place (such as TAI).

However if exact frequency is not critical one could also directly use UT1 instead of UTC (people that use leap second smearing for instance should have no issue with that..).


GPS time is already immune from leap seconds. It has a fixed offset from TAI of 19 seconds and always will.

It also contains UTC offset information so that you can derive the current UTC time from GPS if you wanted to.


As long as everyone agrees on the time GPS will work. It's a question of how much deviance from solar time we are willing to tolerate vs how much effort we want to tolerate.


Instead of inserting a leap second, can't we standardize leap smearing like Google does? https://developers.google.com/time/smear. This will probably also work for negative leaps.


Many computer programs use UTC time (which means you have to deal with leap seconds) when they really should be using TAI (which is based purely on the physical progression of time on the earth's geoid, with no arbitrary input that has to be updated periodically). https://cr.yp.to/proto/utctai.html

Generally, you should only be dealing with UTC for things that have to display/accept a time to/from human users; there's no reason for computers to reason about the interval between events using UTC.


I hear this argument quite a lot. But using TAI is no different from using UTC in terms of complexity because both requires the complex problem of data distribution (either of leap seconds or of UTC-TAI differences). If you ever have to use UTC somewhere you can't escape from that problem; using TAI is only moving the goalpost.


No, using TAI is simpler, because it is really the time that has passed since some origin. Therefore any operation performed with its values behaves as expected.

Neither UTC nor local times are times in the physical sense. At most you can say that they are times passed from a non-constant origin of time, which changes frequently and in an unpredictable mode for the future.

People always forget what UTC and local times really are and they very frequently make mistakes in their handling, especially for future events.

The foolproof way, as advocated by Bernstein and others, is to use only TAI for internal timekeeping and computation, and to convert to UTC or local times only the values that will be seen by human eyes.

I have done like this since many years ago, and it was always simpler for me. It would have been much simpler if this would have been a general practice, as that would have eliminated the useless conversions required for communication with other systems that use UTC, e.g. when using NTP.


Using TAI exclusively is simpler, but that's not the parent advocates (and neither do you, I believe).

Using TAI primarily and UTC in display has two main benefits that I consider marginal. The first is that the addition and subtraction is associative and transitive, but their use towards non-zoned wall clock timestamps like time_t is limited anyway; they normally operate on zoned date & time (e.g. "tomorrow" typically refers to the local time) or on monotonic and steady timestamps with no particular epoch. The second is that the timestamp doesn't repeat but this can also happen with TAI, just that it can only happen with (a large amount of) clock synchronization; you should be prepared of wall clock timestamps repeating no matter it's UTC or TAI.

For those reasons I consider that using TAI primarily and UTC in display is not much superior to using UTC primarily. Not that I don't wish TAI to be universal, but given the current circumstances the switch doesn't justify its cost.


Using TAI is simpler because:

* Many apps can get away with only using TAI, so they don't necessarily have to track the difference between UTC and TAI

* If your system uses TAI in its distributed coordination mechanisms, you don't need to worry about leap seconds and an error in the UTC-TAI difference won't cause your system to fail


> Many apps can get away with only using TAI, so they don't necessarily have to track the difference between UTC and TAI

Most systems don't need to keep the difference between UTC and TAI to display the UTC time anyway. TAI matters when you have to manipulate the non-zoned wall clock (as opposed to date and time components) and that's surprisingly rare.

> If your system uses TAI in its distributed coordination mechanisms, you don't need to worry about leap seconds and an error in the UTC-TAI difference won't cause your system to fail

This is a legitimate concern, but TAI is not the solution (you should probably be using PTP or GPS clock if you do care that much) nor an essential part of the solution (any monotonic clock, including "leap smears" and UTC-SLS, will do fine). In fact I think most perceived benefits of TAI are actually of monotonic clocks and not inherent to TAI, and UTC is already monotonic... (Yes, I'm distinguishing the actual UTC and time_t; for the latter there is a relatively simple conversion to monotonic however.)


Yeah, you're right that the primary benefit of TAI is that it's monotonic - but it's better to have a globally consistent monotonic schelling point than to deal with every machine having its own detached monotonic clock.

> GPS clock

GPS and TAI are essentially the same, modulo a fixed 19 second offset.


This is what I'd _like_ to do, but unfortunately it's not well supported. As far as I know, all OSes store the time including leap seconds, so there's no way to go back to TAI. And even if it were, that time zone is not available in user programs.

edit: Oh, I see that it's available in Linux https://www.man7.org/linux/man-pages/man3/clock_gettime.3.ht... . Can't find similar calls for Windows or OSX


> The acronym TAI refers to International Atomic Time.

Good one. Wikipedia[0] is more helpful and states it comes from "temps atomique international" which is french.

[0]: https://en.wikipedia.org/wiki/International_Atomic_Time


TAI is for keeping track of chronology in this gravity well of ours. UTC is for exposing this as a culturally useful value. UTC uses TAI.


Why is this downvoted? It sounds great to my untrained ear. You could even use TAI for user-facing timestamps (to get consistent arithmetic) and then convert to UTC only in the end before displaying the date!


You know in some places they will forget the conversion so you'll have weird cases that are off by 37 seconds.


Surely that's

a) Discovered sooner than a leap year bug, and b) Given that humans and not machines are the consumers of the UTC time, less consequential than a leap year bug.


That seems like it would be obvious, though, so people would fix it.


This argument naturally leads to the conclusion that you should basically never use UTC. So you use a singularity free coordinate for machine storage of times, TAI, but most users will want those displayed in their local times and nobody is using UTC. UTC is just an intermediate time, computed using leap seconds from TAI that you use on your way to a local time. But you could just remove that step.

Seems sensible to me actually. I don't really see why I would personally want to ever use UTC over TAI or local time. Sadly, a lot of time libraries don't seem to expose TAI as easily as they expose UTC, which I guess is why I continue to use UTC.


Re: the 'negative second' fears - in usual operation, NTP adjusts your local machine downward all the ... time.

There's really nothing to be afraid of; a few services would implement some skew, the rest of us would get NTP updates and life continues.


ntpd can be configured to disallow negative time adjustments, and anything that runs a database definitely should have that configured. The system clock is presumed to always increment for most databases, and screwing with that can cause data loss.


I think the concern is more about whether the negative leap second handling in all of the different, interacting NTP server implementations (which have never been exercised in the wild) will be good, bad or ugly.

If you read around the topic, there was plenty of non-conformance for the relatively recent leap seconds.

As you say, clock skewing happens continuously in NTP so that part looks proven.


What about my DIY double entry book system I wrote using various JavaScript libraries?

I have no clue.


What causes changes in the rate of rotation of the earth?


The angular momentum of the "earth system" is constant [1]. The variation in angular velocity of the solid earth can be ascribed to change in the angular momentum carried by oceans and atmosphere. These changes can be due to change in velocity (winds, currents) or moment of inertia (moisture in atmosphere).

[1] If we ignore friction due to tidal forces from moon and sun (which is valid on these time-scales), the "earth system" is not affected by any non-central forces and so the angular momentum is constant.


Right.

But tidal forces are not completely irrelevant to leap seconds.

The length of the atomic SI second was calibrated to match the previous sidereal second, which was based on the length of a year. In practical terms, the sidereal second was based on Newcomb’s Tables of the Sun, which was a model of the solar system developed at the end of the 1800s based on historical astronomical observations https://en.wikipedia.org/wiki/Newcomb%27s_Tables_of_the_Sun

So the length of the second is based on the length of the year 1900, and derived from older astronomical measurements of the rate of movement of bodies in the solar system.

Leap seconds exist because by the 1960s the length of day no longer matched Newcomb’s calculations for 1900: it was longer by 1 or 2 milliseconds.


> change in the angular momentum carried by oceans and atmosphere

...and some liquid (or at least semi-liquid) stuff beneath the crust?


Is the total mass of all mined minerals and hydrocarbons more than rounding error in the calculation of moment of inertia?

This would assume that mined materials stay at a higher elevation than they were. And that the materials above those items didn't collapse down and/or aren't heavier than the mined mass.


Slowing down of the earth is caused by wind turbines, just as extracting energy from the tides causes the moon to fall towards the earth. When the earth will have completely stopped rotating, and the moon be dangerous close to the earth, that will be yet another shitshow. And they call this "renewable energy"?

P.S.: not serious


Earthquakes, movement of tectonic plates, etc. can shift the mass of the earth closer to or further from center. This speeds up or slows rotation, like when you are spinning in a chair and pull your arms in. Some energy is probably also lost to tidal friction with the moon.


China slowed the rotation of the earth very slightly when they built the Three Gorges Dam.

https://www.kinetica.co.uk/2014/03/27/chinese-dam-slows-down...


people spinning counterclockwise! (https://xkcd.com/162/)


Earthquakes, nuclear explosions, tectonic plate activity, the moon slowly moving away from us amongst other causes.


I don't think nuclear explosions belong in that list. They have tremendous power, but very limited energy on a planetary scale. Do you have a source that says otherwise?


Good question. It means that something is moving mass away from the axis of the Earth.

There are only two candidates I can think of:

1. Movements in magma below the Earth's crust.

2. Global warming. Less ice at the poles means that water has moved away from the Earth's axis, which will slow the rotation down. I haven't run the maths to see if that adds up though.


Just because everyone seems to be complaining about leap seconds: I think they are the right solution for the problem. I do hope we won't need a negative leap second, that would break a lot of code for sure.

Time related stuff sure is messy.


> I do hope we won't need a negative leap second, that would break a lot of code for sure.

Someone ended up asking on one of the FreeBSD mailing lists, and it seems that they test for it:

* https://lists.freebsd.org/pipermail/freebsd-stable/2020-Nove...

So at least when it comes to the FreeBSD kernel and ntpd, there's nothing to worry about there. Userland code may be a different matter of course.


> NTP or kernel timekeeping code expects that it will be an appalling shitshow.

I’m curious what systems would catastrophically fall apart if time went back one second?


A negative leap second (the appalling shitshow from the article) would make us skip a second, though.

Going back one second is what generally happens now when a Linux box does the "inserting leap second" thing. It goes from 23:59:59.999999 to 23:59:59.000000, then runs that whole second again. You get to 23:59:59.999999 again, and then you finally roll over to 00:00:00.000000.

From the perspective of the typical time_t rendering of Unix time, there is no way to uniquely represent that 61st second. It just "disappears".

Having lived through systems going backwards some 17 seconds due to a botched NTP-GPS appliance config, I can tell you that what died at that particular site was all of the locking code that used wall time clocks instead of monotonic clocks. They all CHECKed and died when their preconditions were no longer valid.

This didn't have to happen. They could have used monotonic from the get-go, and then they wouldn't have died when the clocks got yanked backwards 17 seconds to the proper time base.


>'From the perspective of the typical time_t rendering of Unix time, there is no way to uniquely represent that 61st second. It just "disappears".'

This is a great and really intuitive summary of the problem. Is there a certain class of problem related to disappearing second? Like are these more likely to be filesystem issues or things that rely on timestamps? Or are there second order problems as well?

>"I can tell you that what died at that particular site was all of the locking code that used wall time clocks instead of monotonic clocks. They all CHECKed and died when their preconditions were no longer valid. I can tell you that what died at that particular site was all of the locking code that used wall time clocks instead of monotonic clocks. They all CHECKed and died when their preconditions were no longer valid."

Sorry if this is a silly question but was that check simply that "time t1 is greater than time t0"? Also was the duration of that outage(17 seconds) or would this have been equally catastrophic at a single second?


Well, okay, so, the problem is like this: let's say you wanted to schedule something to happen during that exact extra second in the end of June 2015 when we were all standing around watching UTC do its little extra dance. You pick 1435708800. Trouble is, that Unix time applies to both 23:59:59Z and 23:59:60Z on that particular day.

You can't target it beforehand or after. It's just... gone.

It's not a problem from the point of view of programs, since they just got whatever time_t value they got, and they don't know the bigger perspective. It's more of a mapping from outside->in problem.

Put it another way: try writing a program that'll call clock_gettime() and will say a message at a later time you select. You can't put in 23:59:60Z because there's no way to represent it, and indeed, you won't even be able to tell when the time comes unless you special-case it and notice _that particular second_ repeating itself... or reach into the kernel to look at the leap bit, or worse. It's a real time in meatspace, but you can't target it with the tools at hand. That's the problem.

Regarding the 17 second thing, that's because someone decided to switch off the thing which (correctly) applies the adjustment factor to GPS time to make it NTP time. There was a 17 second difference at the time (GPS to NTP), and with it off, we were shipping GPS time to hosts as if it was NTP time.

In theory, any regression of the clock long enough to not let the actual passing of time push it past the sanity check time point in the lock stuff would have caused this. The thing is, a small-scale time step (from ntpd, say) normally happens at boot up, not later, and it's on a system by system basis.

The 17 second excursion happened on hundreds of thousands of machines all at once, and, yeah, it was noticed.


Great explanations. Thanks!


> Having lived through systems going backwards some 17 seconds due to a botched NTP-GPS appliance config

Well then, it really was botched. I was running an off-line hodge podge of 50 or so RH7.3 + Win2K systems back in 2002, where I had to manually fix the few second drift every couple of months, and the RH machines all slewed faster/slower to adapt, non ever went backward or jumped forward (Don't remember how the Win2K handled it; They weren't mission critical and I didn't care much).


I was at Twitter when lots of machines seized.

https://www.wired.com/2012/07/leap-second-glitch-explained/


Distributed alogrithms often depend on precise clocks to maintain consistency. See google spanner for instance.

Updating the clock one second means taking a whole cluster offline untill you are sure the clocks are synchronized exactly before bringing them online again. It brings an outage and potential of headaches.

If the clocks are not synchronized you break the consensus, might cause transactions to commit partially or cause inconsistent reads and writes.


Those algorithms should be using TAI, not UTC.

Any distributed system that breaks in the presence of leap seconds is necessarily poorly designed.

The fact that any non-UI code uses any time standard except TAI is a very unfortunate historical mistake that will hopefully be cleaned up over the next few years.

Thankfully since 2013, Linux kernel has supported TAI.


> The fact that any non-UI code uses any time standard except TAI is a very unfortunate historical mistake that will hopefully be cleaned up over the next few years.

Unfortunately, almost everything is UI code in some way, so this won't be cleaned up.

The problem with TAI is that you cannot convert a future time to UTC if it is farther than 6 months or so in the future (because leap seconds have not been decided), so unless it never needs any conversion to/from UTC, you can't use it. (are logs UI? is crontab UI? eventually everything is)

Furthermore, you can't build an embedded no-need-to-update-every-6-month system with available common protocol - you would need to continuously feed it the list of leap-seconds, for which AFAIK there is no widely deployed protocol.

But if you use UTC, there are many protocols (TIME, NTP, SNTP, PTP), at least one of which you'll use anyway because of clock drifts, that will let you keep working.


> The problem with TAI is that you cannot convert a future time to UTC if it is farther than 6 months or so in the future [...]

Centimeter's argument is that such time should not be converted to TAI in advance, much like the local time for the future point (since the local time zone offset may change). All points on this also apply to the local time, e.g. it's already true that non-updatable embedded systems can't reliably do the local time calculation.


They can’t do local time, but they can do UTC (and do it) without much investment if you run an NTP client. (Have 500 or so such units deployed, SNTP was giving too much trouble, switched to chrony at some point and those units require much less support)


> Thankfully since 2013, Linux kernel has supported TAI.

Only with the correct ntpd configuration [1]. For that reason it is impossible to rely on that API in end-user libraries.

[1] https://superuser.com/questions/1156693/is-there-a-way-of-ge...


And from what I can tell it doesn't work with systemd-timesyncd, which many linux systems are using now.


Pretty sure the Spanner case isn't using either one. Google's systems futz with the speed of time near the leap second to distribute it across a wide span about midnight, UTC.

https://developers.google.com/time/smear


About 10 years ago all Java processes everywhere deadlocked because of this.


Technically, that was a Linux kernel bug that Java happened to be heavily affected by: https://news.ycombinator.com/item?id=4188412


Linux had two different kernel bugs around this, IIRC. MySQL got confused too (although that one just ate cpu until you restarted the daemon, it didn't crash, and it didn't prevent service)


> At the moment the Earth is rotating faster than in recent decades: these shorter days, with a lower length-of-day, means the milliseconds accumulate more slowly, and we get fewer leap seconds.

Does anyone know why Earth's rotation has sped up?


The biggest factor is thought to be core-mantle coupling.

The Earth is built in layers. The solid bit we see is called the crust. Then there is a liquid layer where heat melts rock, that's called the mantle. And then at the center of the Earth the pressure again turns things solid, that's called the core.

All of the things affecting the Earth's rotation in the long term pull on the crust. The two biggest are tides (slows us by 2.3 ms/century) and glacial rebound (estimated to speed us up by 0.7 ms/century). The result is that the crust and the core wind up turning at different rates, and interact with each other through the mantle. Which transfers angular momentum back and forth between the crust and core.

The article that first showed this is https://www.nature.com/articles/333353a0 if you want to look further.


Leap seconds are such a pain. I understand why they cannot be allocated algorithmically but that doesn't make them any easier to deal with.


Aren't leap seconds built into GNSS protocols? i.e. if you have a GNSS receiver, leap seconds will be accounted for in ZDA NMEA sentences.


Yes, the satellites should know 6mo in advance. UTC is still a head ache though.


We should have two “keep on your toes” leap seconds per year, one +1 and one -1 (can it be done ?) in lieu of daylight savings time or together with it for those that still keep it.


'in PostgreSQL you can use “allballs” in a time literal as an alias for midnight'

Learned something new today!


This is great news! No leap second in 2020. It won't be a second longer than necessary.


Aren't the people in charge of UTC discussing never having a leap second ever again?


Monotonically rising time with a negative leapsecond...


Actually the negative leap second is "more monotonic" than the regular one. Normally a leap second adds an extra second (58, 59, 60, 00), but a negative leap second skips one (58, 00).

Uniquely specifying the four times in the first sequence as UNIX timestamps isn't strictly possible, but it is possible with negative leap seconds.


Now this is going to sound crazy but what if we just extend the year by a few seconds by default? Then we can just skip seconds as needed.


phew!




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: