17
$\begingroup$

I noticed that we sometimes call the error terms "innovations". I do not understand if this is in special situations or if these terms can be used one for another. Then, another question is "why do we call error terms "innovations"? thanks

$\endgroup$
3
  • 1
    $\begingroup$ This always frustrated me about econometrics. It always seemed like an attempt to grasp for descriptive meaning where none existed. Same reason, I imagine, that regression is taught misleadingly as "fitted function plus unmodeled error" rather than "estimated conditional mean plus unmodeled variance." I'd love to know if the truth is more sympathetic $\endgroup$ Jan 1, 2015 at 20:13
  • 1
    $\begingroup$ An interesting case arises in relation to two classes of related models: linear innovations state space models (also known as single source of error models) and multiple source of error state space models. Refer to Rob Hyndman's co-authored book on exponential smoothing, for example. $\endgroup$ Jan 2, 2015 at 16:24
  • $\begingroup$ @Graeme Walsh : The distinction between innovations state space models and multiple source or error actually goes even further back to Thomas Kailath's result that a state space model could be written using only one error term ( either in the late 60's or early 70's ), thus avoiding the use of a seperate error term for the state. But, I am not familiar with how or if this "innovations" notation in the KF carried over to its general use in all models. $\endgroup$
    – mlofton
    Dec 12, 2023 at 16:05

2 Answers 2

18
$\begingroup$

The innovations are used in the time series the same way as errors in cross-sectional analysis (such as OLS). For instance if you data generating process is $$y_t=0.9y_{t-1}+\varepsilon_t$$, then we estimated it as $$y_t=0.85y_{t-1}+e_t$$, we call $\varepsilon_t$ innovations (or errors), and $e_t$ - residuals.

For instance, take a look at this MATLAB help page on ARIMA class, where they always refer to innovations in the place where you'd expect to see errors in cross-sectional analysis such as in this MATLAB help page for LinearModel class. In cross-sectional context the model could look like $$y_i=0.9x_i+\varepsilon_i$$

In this MATLAB help page for arima.infer() method, which estimates innovations, the estimated errors are called residuals as usual.

So, I conclude that innovations are ok to interchange with errors. It's called innovations because in time series context the errors bring new information to the system. In cross-sectional context it doesn't make a sense to call them new, as the observations come not in time-ordered sequence. So, observation number 10 is not newer or older than observation number 9. In time series, 10 comes after 9, so in this regard the error/innovation can be seen as a new information from the point of view of the observer who hold the information set up to time 9.

$\endgroup$
6
  • 3
    $\begingroup$ The question asks "why?" $\endgroup$
    – whuber
    Jan 2, 2015 at 16:32
  • 3
    $\begingroup$ OK, so what is your answer to that first part? And then, given your answer (which presumably is no, they are not interchangeable), it would be much more useful to explain why not. Please bear in mind that "innovation" appears to be used in the same context quite generally in time-series analysis, (going back to Shannon and maybe Kolmogorov, neither of whom were specifically interested in economic time series) so it probably is not sufficient to appeal only to the econometrics literature. $\endgroup$
    – whuber
    Jan 2, 2015 at 16:37
  • 1
    $\begingroup$ I agree with @whuber in the need to appeal to the broader literature. As referred to in my comment to the OP, some authors in the state-space modelling literature seem to have used the terms interchangeably. Why they do this, however, remains to me unknown. $\endgroup$ Jan 2, 2015 at 16:45
  • 1
    $\begingroup$ Ok, guys, I re-wrote my answer from time-series perspective. Good feedback from both. $\endgroup$
    – Aksakal
    Jan 2, 2015 at 17:11
  • 4
    $\begingroup$ +1 I'm upvoting in recognition of and thanks for the additional research you have performed and your thoughtful presentation of it. However, I am still wondering whether some authorities--including MatLab--might be trying to make some subtle distinctions in their uses of "innovation," "error," and "residual." It might be interesting to keep pursuing this... . $\endgroup$
    – whuber
    Jan 2, 2015 at 17:14
0
$\begingroup$
  1. See Lawrence Christiano''s "Brief Review of VAR's" notes. He distinguishes between u(t)'s which are new errors that occur at each time in the time series (t,t+1) but do not necessarily mean anything, with underlying economic shocks which MAY persist on an Impulse response graph or show a pattern at some level "C", e.g. ut=C*et.
  2. If there is no impact in the Long run, C=0, and ut=0 means random Gaussian white noise with E[ut]=0 mean. But if for example supply shocks are persistent, they will show some impulse response pattern long run where C has some value or level, like may C=.90% of the error ut will impact demand each time.
  3. At least that's how I read it. But agree sometimes used interchangeably.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.