Given a time series, I'd like to estimate the parameters of an AR(1) model for it. As explained on wikipedia, there are different ways for doing that. What may be called a naive method is to compute the sample mean, variance, and autocovariance of the sample and then obtain the parameters of the AR(1) model using some simple equations. Alternatively, one can use more complicated things like maximum-likelihood estimation.

What is the benefit of one method versus the other?

Are there any provable differences? For example, suppose my data really comes from an AR(1) process, is the "naive" method provably less accurate than others such as maximum-likelihood?


is the "naive" method provably less accurate than others such as maximum-likelihood?

Under the assumptions of the model, then basically, yes, the naive approach is less accurate, at least in some senses, via considerations such as the Cramer-Rao lower bound and the Rao-Blackwell theorem.

There are a variety of concepts and theorems you may need to investigate to get a reasonably complete picture, but there's a number of senses in which MLEs are often 'good', at least asymptotically.

That said, in the case of AR models, there are much simpler approaches which are asymptotically equivalent to ML (such as conditioning on the first $p$ obervations in an AR(p)), so in that case there may not be much to choose between them.

  • $\begingroup$ Thanks. Can you elaborate/point to the quantitative difference between the naive and ML/other methods? Can you also say more about simple approaches, I am not sure what you mean by "such as conditioning..."? $\endgroup$ – Manu Jun 2 '13 at 17:02
  • $\begingroup$ Sorry, I am not clear what you seek with your first question. On the second part, if you condition on the first p observations, AR(p) just becomes like a straight Gaussian regression problem; it's what I'd do 'naively'. $\endgroup$ – Glen_b -Reinstate Monica Jun 3 '13 at 3:28
  • $\begingroup$ I seek an error measure with respect to which maximum likelihood provably behaves better than what I called the naive method. (Otherwise, it is not clear to me what "less accurate" means.) Do you have a pointer for the straight Gaussian regression problem? Thanks! $\endgroup$ – Manu Jun 3 '13 at 15:10
  • $\begingroup$ "Accurate" is for you to define, since it's your question. But the usual ones at the links I referred to in my answer are common (the links mention comparing variance and MSE, I believe). $\endgroup$ – Glen_b -Reinstate Monica Jun 3 '13 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.