Handle leap seconds in time series data

Question

I am trying to figure out how to handle time series data.

Initially, I thought it was self-evident (for my purposes) to only allow non-overlapping data blocks. (I.e., non-overlapping, time wise.)

Then it occurred to me, what about leap seconds? There could be a leap second in there, making one of the hours 3601 seconds long instead of 3600, and throw the whole thing off.

How do people handle things like these? Shaving off the last second? Have a "fuzz" factor and allow a second up or down? Am I over-thinking this?

Answer 1

The industrial-strength solution — which might not be of interest in your particular case! — is to use a time standard that experiences no leap seconds such as TT, Terrestrial Time, and then translate between the clock values you need on input and output into your internal TT representation.

http://en.wikipedia.org/wiki/Terrestrial_Time

A library that might help in converting would be:

http://www.usno.navy.mil/USNO/astronomical-applications/software-products/novas/novas-python

But if you have anchored your data against the times returned by the Python time module then you will need to look into the meaning of Unix time seconds, which is well-described by the answer that someone already linked to above:

Is time() guaranteed to be leap-second aware?

Answer 2

Yes I think you may be worrying a lot about nothing. In time series the important time unit is the increment of time between two points in the series. Most time series analyses deal with equally spaced data. If the unit of increment is one second then it is possible that the minute containing the leap second is a little different because of the extra second. But more often the incremetn will be hours, days, weeks, months or years. In any of these cases that extra leap second in one of the time intervals really means nothing.

Answer 3

Your idea that the blocks of time should be 'non-overlapping' is basically right. It's not clear what kind of data you're talking about. But for example, say you were examining variation in rainfall over time. You measured the rainfall at hourly intervals. If you really thought one second's worth of extra rainfall would introduce an intolerable error into your research -- and that this error wouldn't be swamped in any case by other much larger sources of measurement error -- then you would be sure to define one hour as 3600 seconds and not with reference to coordinated universal time (UTC), which inserts leap seconds.

Alternatively, if you were somehow tied to defining time with reference to UTC, then you could make a correction to the hour containing the leap second, probably multiplying your measurement by (3600 / 3601) (which would only reduce your measurement by 0.03%).

But yes, you are probably over-thinking this! Leap seconds only occur less than once a year on average, and will be ignored by timekeeping devices that aren't linked up to a UTC source.