Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm having some problems with this question, and was hoping someone here could help.

Let $X_1,\ldots,X_2$ be $n$ determinations of a physical constant $\theta$. Consider the model

$X_i = \theta + e_i,$ $~~~~~~i = 1,...n$

and assume

$e_i = \alpha e_{i-1} + \beta e_{i-2} + \epsilon_i$, $~~~~i = 1,\ldots,n$, $~~~e_0=0, e_{-1}=0$

with $\epsilon_i$'s i.i.d standard normal, and $\alpha$ and $\beta$ are known constants. What is the maximum likelihood estimate of $\theta$? Carefully justify each step of your derivation.

share|improve this question

See whether you can justify the steps of this strategy and carry it out.

  1. The vector $(X_i)$ has a multivariate Normal distribution. (Proof?) Therefore we need to find the parameters of this distribution, which are its mean and covariance matrix.

  2. All the $X_i$ have a common expectation of $\theta$ so the multivariate mean is the vector $\mu = (\theta, \theta, \ldots, \theta)$.

  3. The covariance matrix $\Sigma$ can be computed using the recurrence relation for the $e_i$. To do this, express the $(i, i+j)$ entry in this matrix for $j\ge 0$ (which is Cov$(X_i,X_{i+j})$) in terms of the $(i, i+j-2)$ and $(i,i+j-1)$ entries and solve the resulting linear recurrence relations. The solution, as a function of $i$ and $j$, will otherwise depend only on the known values $\alpha$ and $\beta$

    (You might want to start by working out formulas for the diagonal entries Var$(X_i,X_i)$.)

  4. The likelihood of the data $\mathrm{X} = (X_1, X_2, \ldots)$ is the probability density of a multivariate Normal distribution with parameters $(\mu,\Sigma)$ evaluated at $\mathrm{X}$, considered as a function of the unknown $\theta$, but depending explicitly on whatever values $\alpha$ and $\beta$ might be known to have.

  5. The log likelihood is a quadratic function of $\theta$. Using a convenient formula for the locations of minima of quadratic functions (aka vertices of parabolas), write down the maximum likelihood estimate of $\theta$.

share|improve this answer
Thanks for that. I spent some time trying to work it out, but I'm still having trouble with step (3). I tried following your suggestions but, I can't seem to be able to write down the covariance matrix. Any help would be much appreciated! – Just Jul 21 '14 at 23:02

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.