Is there a way to recover a temporal dependence structure in a time series from a regression against time?

Question

Consider a time series:

$X_1,X_2,...X_{n-1},X_n$

This series can also be written as a function of time $X(t)$, so that:

$X_1,X_2,...X_{n-1},X_n = X(t_1),X(t_2),...X(t_{n-1}),X(t_n)$

Most forecasting methods, such as ARIMA or Exponential Smoothing, will try to model the time series based on some temporal dependence structure. That is, they will try to find a mapping between future values and past values of the series, which can be written as:

$\hat{X_i} = f(X_{i-1},X_{i-2},...)$

The idea here is that if we want to predict a time series based on it's historical data, then we want to discover the dependence structure between past and future values.

Another approach, not as commonly used, is to model the time series directly as a function of time:

$\hat{X}(t) = f(t)$

With $f(t)$ determined by some (linear or non-linear) regression against time itself as a variable (instead of as a function of past values).

This is the approach that is used, for example, by the Facebook Prophet algorithm.

Is there any way of recovering the first type of model from the second type?

If I have managed to find a good model of the type:

$\hat{X}(t) = f(t)$

Is there any way to go from $\hat{X}(t) = f(t)$ $\rightarrow$ $\hat{X}_i = f(X_{i-1},X_{i-2},...)$ ?

Answer 1

It is important to distinguish between data generating process and mathematical relationship. It may be possible that there is a mapping (and possibly non-unique), $\hat{X}(t) = f(t)$ $\rightarrow$ $\hat{X}_i = f(X_{i-1},X_{i-2},...)$. However, this does not mean that both can be considered as same data generating process.

When we model a time series (or a process) by $\hat{X}_i = f(X_{i-1},X_{i-2},...)$, we assume that each new observation by design is generated by this process. In polynomial modeling the innovations from previous periods play no role in influencing the realized value of current period. In dependence structure modeling, innovations from previous periods are directly part of current observation. So you see, there is a very significant difference in data generation process.

On the other hand, there may be a mathematical relationship that can give a non-unique mapping. Consider this:

$\hat{X}(t) = a_0+a_1t+a_2t^2$

$\implies \hat{X}(t-1) = a_0+a_1(t-1)+a_2(t-1)^2$

$\implies \hat{X}(t-1) = \hat{X}(t)-a_1+a_2-2a_2t$

$\implies \Delta\hat{X}(t) \equiv \hat{X}(t)-\hat{X}(t-1) = a_1-a_2+2a_2t$

$\implies \Delta\hat{X}(t)-\Delta\hat{X}(t-1)=2a_2$

Therefore, $\hat{X}(t)=2\hat{X}(t-1)-\hat{X}(t-2)+2a_2$

So, from $\hat{X}(t) = f(t)$, we have found $\hat{X}_i = f(X_{i-1},X_{i-2})$.

What's fishy in this? We have actually found $\hat{X}_i = f(\hat{X}_{i-1},\hat{X}_{i-2})$. But interestingly, we can still model $\hat{X}(t) = a_0+a_1t+a_2t^2$ by $X(t)=2X(t-1)-X(t-2)+\epsilon_t$. Just the innovations will be completely different now. Further, the latter relationship is will hold irrespective of the value of $a_0$ and $a_1$. So the relationship will not be unique.

Answer 2

Great Question:

There is nonsense and there is nonsense but the most non-sensical nonsense of them all is statistical nonsense as promoted by PROPHET promoting polynomials in time rather than level\step shifts (intercept changes ) and time trends with possible break points. Please see my answer/comments Why is my high degree polynomial regression model suddenly unfit for the data? and furthermore for an intelligent assessment of anachronistic polynomial fitting see @huber's insightful reflections in Does the p-value in the incremental F-test determine how many trials I expect to get correct? .

Forming an ARIMA model with possible deterministic trends and/or levels is much more approriate and legions beyond Prophet's capabilities. I have fully researched Prophet and on my opinion find the only thing of value is the creative choice of the name. Their treatment of daily data is particularly wanting.

ANSWER : Not to my knowledge since any sufficient ARIMA model might contain not only ARIMA structure bur Pulses , Level/Step shifts , Seasonal Pulses and deterministic time trends.

Answer 3

Have you tried asymmetric eigenvector maps (AEM)? It is useful for summarizing temporal auto-correlation in orthogonal vectors that can be used as predictive variables. Also, you have the same approach for spatial auto-correlation but is called Moran Eigenvector maps (MEM). I hope this will help you,

Best José.