7
$\begingroup$

What are the relation and differences between time series and linear regression?

I have a strong grasp of linear regression, and a beginner's grasp on time series analysis; I know the Box-Jenkins method and understand the concepts. To solidify this understanding, I would like to compare and contrast the two methods to understand if time series analysis is an extension of linear regression.

Maybe the best way to answer this question is to compare and contrast the model assumptions of each method. Does time series analysis share all assumptions of linear regression, with a few extra assumptions added in (related to autocorrelation, stationarity, etc.)?

Note: This question has been asked here but the answers go off-topic and discuss the flaws of a Cornell professor's understanding of time series analysis. I do not have enough reputation to comment on that thread.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ autobox.com/dave/regvsbox.pdf discusses issues/differences/opportunities/pitfalls when dealing with time series that your possible regression solutions may be ignoring. It emphasizes the critical assumption of independent observations inherent in regression. $\endgroup$
    – IrishStat
    Commented Feb 15, 2018 at 1:03
  • 1
    $\begingroup$ Thank you for the direct link; I had trouble finding this document through the link posted on the related thread. From your document, it sounds like the two methods are similar enough that one can account for the qualities typical of time series data (e.g. incorporate time and/or back-shifted Y as regressors, etc.) and create a linear regression model. This seems to point to time series analysis being a more complex extension of linear regression. Would you agree? And does this mean that time series analysis shares the assumptions of linear regression, plus some? $\endgroup$
    – Renel Chesak
    Commented Feb 15, 2018 at 1:31
  • 1
    $\begingroup$ time series is more complex/correct as more assumptions can be tested culminating in a more appropriate model. $\endgroup$
    – IrishStat
    Commented Feb 15, 2018 at 10:31
  • 1
    $\begingroup$ If we understand the "regression method" in the thread you reference as being one particular idiosyncratic approach, in contrast to your use of "linear regression" in a standard sense, then I agree you are asking a different question. (+1) $\endgroup$
    – whuber ♦
    Commented Feb 15, 2018 at 17:16

2 Answers 2

Reset to default
2
$\begingroup$

In the context of Statistics, linear regression is solved by maximizing the likliehood that the error of a model linear in basis is the mean of a Normal Distribution. During maximization we assume the observations are independently and identically distributed, clearly not a reasonable assumption for times series data.

Improve this answer
$\endgroup$
1
$\begingroup$

From Ordinary Regression to Time Series Regression:

The time series regression model is an extension of the ordinary regression model in which the following conditions exist:

 Variables are observed in time.

 Autocorrelation is allowed.

 The target variable can be influenced by past values of inputs.

Source: DePaul University lecture slides for CSC 425

I think this answer is lacking in complete details, but is not wrong. @IrishStat gave a link to a document that covers the differences well. Together, these answer the first part of the original question.

I am still looking for answers to the latter half: does time series analysis share the assumptions of linear regression, plus some? For example, linear regression has multiple assumptions about X regressors such as no multicollinearity, linear relationship (correlation) to Y, the X regressors and model residuals are uncorrelated, etc. Do all of these still apply in time series analysis? If we could make a complete list of assumptions that these two methods share, that would be extremely helpful. Thanks everyone!

Improve this answer
$\endgroup$
1
  • $\begingroup$ Fir time series ...1)the X regressors and model residuals are uncorrelated contemporaneously & for all lags 2) there are no pulses , level shifts , seasonal pulses , local time trends and seasonal pulses in the model residuals. 3) the model's parameters are invariant over time 4) the variance of the residuals is homogeneous over time $\endgroup$
    – IrishStat
    Commented Feb 16, 2018 at 20:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.