# Is there a way to recover a temporal dependence structure in a time series from a regression against time?

Consider a time series:

$X_1,X_2,...X_{n-1},X_n$

This series can also be written as a function of time $X(t)$, so that:

$X_1,X_2,...X_{n-1},X_n = X(t_1),X(t_2),...X(t_{n-1}),X(t_n)$

Most forecasting methods, such as ARIMA or Exponential Smoothing, will try to model the time series based on some temporal dependence structure. That is, they will try to find a mapping between future values and past values of the series, which can be written as:

$\hat{X_i} = f(X_{i-1},X_{i-2},...)$

The idea here is that if we want to predict a time series based on it's historical data, then we want to discover the dependence structure between past and future values.

Another approach, not as commonly used, is to model the time series directly as a function of time:

$\hat{X}(t) = f(t)$

With $f(t)$ determined by some (linear or non-linear) regression against time itself as a variable (instead of as a function of past values).

This is the approach that is used, for example, by the Facebook Prophet algorithm.

Is there any way of recovering the first type of model from the second type?

If I have managed to find a good model of the type:

$\hat{X}(t) = f(t)$

Is there any way to go from $\hat{X}(t) = f(t)$ $\rightarrow$ $\hat{X}_i = f(X_{i-1},X_{i-2},...)$ ?

Great Question:

There is nonsense and there is nonsense but the most non-sensical nonsense of them all is statistical nonsense as promoted by PROPHET promoting polynomials in time rather than level\step shifts (intercept changes ) and time trends with possible break points. Please see my answer/comments Why is my high degree polynomial regression model suddenly unfit for the data? and furthermore for an intelligent assessment of anachronistic polynomial fitting see @huber's insightful reflections in Does the p-value in the incremental F-test determine how many trials I expect to get correct? .

Forming an ARIMA model with possible deterministic trends and/or levels is much more approriate and legions beyond Prophet's capabilities. I have fully researched Prophet and on my opinion find the only thing of value is the creative choice of the name. Their treatment of daily data is particularly wanting.

ANSWER : Not to my knowledge since any sufficient ARIMA model might contain not only ARIMA structure bur Pulses , Level/Step shifts , Seasonal Pulses and deterministic time trends.

• I have played with prophet using the M3 & M4 datasets to test its performance and I did not find it something jaw-dropping either. Nevertheless I think you are over-acting. Yes, it is most probably overhyped but there is no reason to get personal. For example, both of his authors have PhD in analytical disciplines so surely they are not code-monkeys. – usεr11852 says Reinstate Monic Apr 13 '18 at 22:03
• Can you please give a more substantiated view as to what these "incorrect approaches" are and why they are incorrect? (I do value your expertise on the matter so I am curious about your thoughts on this.) – usεr11852 says Reinstate Monic Apr 13 '18 at 22:09
• Not intended to be personal but nobody I know has written about using polynomials in any preferred time series journal in a very very long time. It is still include in textbooks and authors need to sell their textbooks and teachers familiar with "older methods" need to have printed material to use as the basis for their lectures on the way things were done in the olden days. In terms of incorrect aproaches one would start with using only one approach and not using the data to sort out what alternative approaches would suggest. – IrishStat Apr 13 '18 at 22:17
• As a practitioner/progammer and older statistician , I grew up with fitting higher polynomials. Part of my education I was to investigate what happens if you for example fit higher and higher polynomials to say 4 values, The conclusion is fit a cubic it will give you an r square of 1.0 . Fitting "aint everything " it is cracked up to be. Model formulation speaks to necessity and suficiency via diagnostic checking. Huber's reflections are from a statistician/mathematician who is not a time series specialist but wise nonetheless.Tks for your nice words, it is nice to know their is an audience, – IrishStat Apr 13 '18 at 22:26
• It is late and I can't really recall my original notes on prophet but basically using Fourier polynomials is not the end of the world when it comes to seasonality... Anyway, as mentioned based on my experience and some quick evaluations like the one here it seems that prophet is far from a time-series answer to all (daily) data but OK... I think I see vaguely your general point. – usεr11852 says Reinstate Monic Apr 13 '18 at 22:41

It is important to distinguish between data generating process and mathematical relationship. It may be possible that there is a mapping (and possibly non-unique), $$\hat{X}(t) = f(t)$$ $$\rightarrow$$ $$\hat{X}_i = f(X_{i-1},X_{i-2},...)$$. However, this does not mean that both can be considered as same data generating process.

When we model a time series (or a process) by $$\hat{X}_i = f(X_{i-1},X_{i-2},...)$$, we assume that each new observation by design is generated by this process. In polynomial modeling the innovations from previous periods play no role in influencing the realized value of current period. In dependence structure modeling, innovations from previous periods are directly part of current observation. So you see, there is a very significant difference in data generation process.

On the other hand, there may be a mathematical relationship that can give a non-unique mapping. Consider this:

$$\hat{X}(t) = a_0+a_1t+a_2t^2$$

$$\implies \hat{X}(t-1) = a_0+a_1(t-1)+a_2(t-1)^2$$

$$\implies \hat{X}(t-1) = \hat{X}(t)-a_1+a_2-2a_2t$$

$$\implies \Delta\hat{X}(t) \equiv \hat{X}(t)-\hat{X}(t-1) = a_1-a_2+2a_2t$$

$$\implies \Delta\hat{X}(t)-\Delta\hat{X}(t-1)=2a_2$$

Therefore, $$\hat{X}(t)=2\hat{X}(t-1)-\hat{X}(t-2)+2a_2$$

So, from $$\hat{X}(t) = f(t)$$, we have found $$\hat{X}_i = f(X_{i-1},X_{i-2})$$.

What's fishy in this? We have actually found $$\hat{X}_i = f(\hat{X}_{i-1},\hat{X}_{i-2})$$. But interestingly, we can still model $$\hat{X}(t) = a_0+a_1t+a_2t^2$$ by $$X(t)=2X(t-1)-X(t-2)+\epsilon_t$$. Just the innovations will be completely different now. Further, the latter relationship is will hold irrespective of the value of $$a_0$$ and $$a_1$$. So the relationship will not be unique.