Timeline for What's wrong if I fit the auto-regression with OLS?
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 15 at 8:23 | comment | added | Sextus Empiricus | A question where the likelihood for an AR1 process is completely spelled out is: How to estimate maximum liklihood of a custom log likelihood function?. If it wasn't for the first term, then the likelihood would be entirely equivalent to OLS. | |
| Nov 18, 2015 at 9:28 | comment | added | Richard Hardy | Thank you for the material. Now I see what I was missing. In an AR(1) example without intercept, I did not consider the contribution of $y_1$ to the likelihood -- I did not realize that it was informative. Intuitively, I thought that the process we started observing at time $t=1$ did not actually launch as a zero-mean realization of a random variable independent of the past; I thought of an infinite past (t=0,-1,-2, $\dotsc$) that we cannot observe. But I failed to realize that given the structure of the process, $y_1$ contributes with its unconditional density to the likelihood. Thanks again! | |
| Nov 17, 2015 at 21:35 | history | edited | Glen_b | CC BY-SA 3.0 |
added 159 characters in body
|
| Nov 17, 2015 at 21:25 | comment | added | Glen_b | @RIchard I was just googling for something to link you to for the algebraic version. This document in section 2 shows it for the AR(1) case; the AR(p) case works the same way. | |
| Nov 17, 2015 at 21:22 | comment | added | Richard Hardy | Thanks for the explanation! Apparently, my confusion is in terminology. Also, I did not make the connection between OLS and maximum likelihood (I only thought in terms of OLS), which apparently explains conditioning. It still gives me some thought to digest. | |
| Nov 17, 2015 at 21:22 | comment | added | Glen_b | ctd... if you write the likelihood conditional on $y_1,...,y_p$ the second term drops out and you're left with the regression model). | |
| Nov 17, 2015 at 21:18 | comment | added | Glen_b | @RIchardHardy you just exactly described why it is conditioning on the first $p$ values. They're removed from $y$, but they're all still in $X$ in the regression (and in regression, $y$ is conditioned on $X$). ergo the likelihood you maximize is one conditioning on the first $p$ y-values. (You can also show it algebraically by decomposing the likelihood, it's quite straightforward. The likelihood for the AR can be split via the prediction error decomposition into two components, that for $y_{p+1},...,y_n$ and $y_1,...,y_p$, ...ctd | |
| Nov 17, 2015 at 18:34 | comment | added | Richard Hardy | I thought that estimating by OLS would amount to effectively cutting the sample by $p$ observations so that lagged regressors could be obtained -- instead of appending the data with some arbitrary values for the negative lags. As such there would be no conditioning on the first $p$ values. | |
| Nov 17, 2015 at 13:09 | history | answered | Glen_b | CC BY-SA 3.0 |