Is there any gold standard for modeling irregularly spaced time series?

Question

In field of economics (I think) we have ARIMA and GARCH for regularly spaced time series and Poisson, Hawkes for modeling point processes, so how about attempts for modeling irregularly (unevenly) spaced time series - are there (at least) any common practices?

(If you have some knowledge in this topic you can also expand the corresponding wiki article.)

I see irregular time series simply as series of pairs (value, time_of_event), so we have to model not only value to value dependencies but also value and time_of_event and timestamps themselves.

Edition (about missing values and irregular spaced time series) :

Answer to @Lucas Reis comment. If gaps between measurements or realizations variable are spaced due to (for example) Poisson process there is no much room for this kind of regularization, but it exists simple procedure : t(i) is the i-th time index of variable x (i-th time of realization x), then define gaps between times of measurements as g(i)=t(i)-t(i-1), then we discretize g(i) using constant c, dg(i)=floor(g(i)/c and create new time series with number of blank values between old observations from original time series i and i+1 equal to dg(i), but the problem is that this procedure can easily produce time series with number of missing data much larger then number of observations, so the reasonable estimation of missing observations' values could be impossible and too large c delete "time structure/time dependence etc." of analysed problem (extreme case is given by taking c>=max(floor(g(i)/c)) which simply collapse irregularly spaced time series into regularly spaced

Edition2 (just for fun): Image accounting for missing values in irregularly spaced time series or even case of point process.

Answer 1

If the observations of a stochastic process are irregularly spaced the most natural way to model the observations is as discrete time observations from a continuous time process.

What is generally needed of a model specification is the joint distribution of the observations $X_{1}, \ldots, X_n$ observed at times $t_1 < t_2 < \ldots < t_n$, and this can, for instance, be broken down into conditional distributions of $X_{i}$ given $X_{i-1}, \ldots, X_1$. If the process is a Markov process this conditional distribution depends on $X_{i-1}$ $-$ not on $X_{i-2}, \ldots, X_1$ $-$ and it depends on $t_i$ and $t_{i-1}$. If the process is time-homogeneous the dependence on the time points is only through their difference $t_i - t_{i-1}$.

We see from this that if we have equidistant observations (with $t_i - t_{i-1} = 1$, say) from a time-homogeneous Markov process we only need to specify a single conditional probability distribution, $P^1$, to specify a model. Otherwise we need to specify a whole collection $P^{t_{i}-t_{i-1}}$ of conditional probability distributions indexed by the time differences of the observations to specify a model. The latter is, in fact, most easily done by specifying a family $P^t$ of continuous time conditional probability distributions.

A common way to obtain a continuous time model specification is through a stochastic differential equation (SDE) $$dX_t = a(X_t) dt + b(X_t) dB_t.$$ A good place to get started with doing statistics for SDE models is Simulation and Inference for Stochastic Differential Equations by Stefano Iacus. It might be that many methods and results are described for equidistant observations, but this is typically just convenient for the presentation and not essential for the application. One main obstacle is that the SDE-specification rarely allows for an explicit likelihood when you have discrete observations, but there are well developed estimation equation alternatives.

If you want to get beyond Markov processes the stochastic volatility models are like (G)ARCH models attempts to model a heterogeneous variance (volatility). One can also consider delay equations like $$dX_t = \int_0^t a(s)(X_t-X_s) ds + \sigma dB_t$$ that are continuous time analogs of AR$(p)$-processes.

I think it is fair to say that the common practice when dealing with observations at irregular time points is to build a continuous time stochastic model.

Answer 2

For irregular spaced time series it's easy to construct a Kalman filter.

There is a paper how to transfer ARIMA into state space form here

And one paper that compares Kalman to GARCH here$^{(1)}$

$(1)$ Choudhry, Taufiq and Wu, Hao (2008)
Forecasting ability of GARCH vs Kalman filter method: evidence from daily UK time-varying beta.
Journal of Forecasting, 27, (8), 670-689. (doi:10.1002/for.1096).

Answer 3

When I was looking for a way to measure the amount of fluctuation in irregularly sampled data I came across these two papers on exponential smoothing for irregular data by Cipra [1, 2 ]. These build further on the smoothing techniques of Brown, Winters and Holt (see the Wikipedia-entry for Exponential Smoothing), and on another method by Wright (see paper for references). These methods do not assume much about the underlying process and also work for data that shows seasonal fluctuations.

I don't know if any of it counts as a 'gold standard'. For my own purpose, I decided to use two way (single) exponential smoothing following Brown's method. I got the idea for two way smoothing reading the summary to a student paper (that I cannot find now).

Answer 4

The analysis of irregularly sampled time series can be tricky, as there aren't many tools available. Sometimes the practice is to apply regular algorithms and hope for the best. This isn't necessarily the best approach. Other times people try to interpolate the data in the gaps. I have even seen cases where gaps are filled with random numbers which have the same distribution as the known data. One algorithm specifically for irregularly sampled series is the Lomb-Scargle Periodogram which gives a periodogram (think power spectrum) for unevenly sampled time series. Lomb-Scargle doesn't require any "gap conditioning".

Answer 5

If you want a "local" time-domain model -- as opposed to estimating correlation functions or power spectra), say in order to detect and characterize transient pulses, jumps, and the like -- then the Bayesian Block algorithm may be useful. It provides an optimal piecewise constant representation of time series in any data mode and with arbitrary (unevenly) spaced sampling. See

"Studies in Astronomical Time Series Analysis. VI. Bayesian Block Representations," Scargle, Jeffrey D.; Norris, Jay P.; Jackson, Brad; Chiang, James, Astrophysical Journal, Volume 764, 167, 26 pp. (2013). http://arxiv.org/abs/1207.5578

Answer 6

In spatial data analysis data is most of the time sampled irregularly in space. So one idea would be to see what is done there, and implement variogram estimation, kriging, and so on for one-dimensional "time" domain. Variograms could be interesting even for regularly spaced time series data, as it has diferent properties from the autocorrelation function, and is defined and meaningful even for non-stationary data.

Here is one paper (in spanish) and here another one.

Answer 7

This is too long for a comment, but I believe it's an important comment here. What is discussed here is a mathematical approach under some specific assumptions on the process being measured, but we can have time series data that do not follow these assumptions!

The modelling approach must answer the question of «why is the time series irregular?» and respond to it correctly. Some potential answers:

• Observations missing at random (e.g. we observe a process in social sciences and we just can't sample it often enough) → approaches based on approximation at regular timestamps, e.g. Kalman filter, are fine and simplify the problem a lot.
• Series of externally observable events (e.g. loan payments) → we need to model spacing explicitly, because shorter or longer intervals will have meaning.
• Series of observable events where the act of observation changes the object (e.g. modelling spaced repetition methods or… some surveys) → can of worms.