10
$\begingroup$

Consider an AR($p$) model (assuming zero mean for simplicity):

$$ x_t = \varphi_1 x_{t-1} + \dotsc + \varphi_p x_{t-p} + \varepsilon_t $$

The OLS estimator (equivalent to the conditional maximum likelihood estimator) for $\mathbf{\varphi} := (\varphi_1,\dotsc,\varphi_p)$ is known to be biased, as noted in a recent thread.

(Curiously, I could neither find the bias mentioned in Hamilton "Time Series Analysis" nor in a few other time series textbooks. However, it can be found in various lecture notes and academic articles, e.g. this.)

I was not able to find out whether the exact maximum likelihood estimator of AR($p$) is biased or not; hence my first question.

  • Question 1: Is exact maximum likelihood estimator of AR($p$) model's autoregressive parameters $\varphi_1,\dotsc,\varphi_p$ biased? (Let us assume the AR($p$) process is stationary. Otherwise the estimator is not even consistent, since it is restricted in the stationary region; see, e.g., Hamilton "Time Series Analysis", p. 123.)

Also,

  • Question 2: Are there any reasonably simple unbiased estimators?
$\endgroup$
  • 1
    $\begingroup$ I'm quite sure the ML estimator in an AR(p) is biased (the existence of the stationarity boundary suggests that it will be biased) but I don't have a proof for you right now (most ML estimators are biased in any case, but we have a little more than that to go on here). [Personally I don't see unbiasedness as a particularly useful property to have, at least in general -- it's like the old joke about statisticians going duck hunting. Ceteris paribus, having it is better than not, of course, but in practice the ceteris are never paribus. It's an important concept though. ] $\endgroup$ – Glen_b -Reinstate Monica Nov 20 '15 at 0:08
  • $\begingroup$ I thought unbiasedness would be desirable when working in small samples, and I have just faced such an instance. In my understanding, in that case unbiasedness was more desirable than, say, efficiency as long as efficiency could be quantified. $\endgroup$ – Richard Hardy Nov 20 '15 at 6:50
  • $\begingroup$ Where bias may not be small (as in small samples), I'd really tend to look for something more like minimum mean square error. What's the point in caring that your estimate could be wrong on average, when in fact your alternative estimate could be much more wrong because it's got a high variance? e.g. if my bias at this sample size for this $\phi$ is 0.1 that might be worryingly large so you'd say "let's use an unbiased estimator"... but if the standard error is large enough that my estimate is typically even further from the correct value ... am I better off? ...ctd $\endgroup$ – Glen_b -Reinstate Monica Nov 20 '15 at 7:14
  • $\begingroup$ ctd. ... I don't think so (not for my usual purposes at least, and I've almost never seen a good argument for unbiasedness in a practical situation that something more like MMSE wouldn't be better for). I care about how wrong this estimate is -- how far I may be off the true value -- not how much the shift in average is if I'm in this situation a million more times. The main practical value in working out the bias tends to be seeing whether you can reduce it easily without much impacting the variance. $\endgroup$ – Glen_b -Reinstate Monica Nov 20 '15 at 7:18
  • $\begingroup$ Good argument, thank you. I will think more about it. $\endgroup$ – Richard Hardy Nov 20 '15 at 7:23
1
$\begingroup$

This is of course not a rigorous answer to your question 1, but since you asked the question in general, evidence for a counterexample already indicates that the answer is no.

So here is a little simulation study using exact ML estimation from arima0 to argue that there is at least one case where there is bias:

reps <- 10000
n <- 30
true.ar1.coef <- 0.9

ar1.coefs <- rep(NA, reps)
for (i in 1:reps){
  y <- arima.sim(list(ar=true.ar1.coef), n)
  ar1.coefs[i] <- arima0(y, order=c(1,0,0), include.mean = F)$coef
}
mean(ar1.coefs) - true.ar1.coef
$\endgroup$
  • $\begingroup$ +1 and thank you! $\endgroup$ – Richard Hardy Oct 11 at 8:02
-1
$\begingroup$

I happen to be reading the same book that you are reading and found the answer to both of your questions.

The biasness of the autoregression betas is mentioned in the book on page 215.

The book also mentions a way to correct the bias on page 223. The way to proceed is through a iterative two step approach.

Hope this helps.

$\endgroup$
  • $\begingroup$ Per the site's guidelines, answers should not simply consist of references to material elsewhere. $\endgroup$ – Alexis Oct 9 '17 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.