# Question with MLE

I'm having some problems with this question, and was hoping someone here could help.

Let $X_1,\ldots,X_2$ be $n$ determinations of a physical constant $\theta$. Consider the model

$X_i = \theta + e_i,$ $~~~~~~i = 1,...n$

and assume

$e_i = \alpha e_{i-1} + \beta e_{i-2} + \epsilon_i$, $~~~~i = 1,\ldots,n$, $~~~e_0=0, e_{-1}=0$

with $\epsilon_i$'s i.i.d standard normal, and $\alpha$ and $\beta$ are known constants. What is the maximum likelihood estimate of $\theta$? Carefully justify each step of your derivation.

-

See whether you can justify the steps of this strategy and carry it out.

1. The vector $(X_i)$ has a multivariate Normal distribution. (Proof?) Therefore we need to find the parameters of this distribution, which are its mean and covariance matrix.

2. All the $X_i$ have a common expectation of $\theta$ so the multivariate mean is the vector $\mu = (\theta, \theta, \ldots, \theta)$.

3. The covariance matrix $\Sigma$ can be computed using the recurrence relation for the $e_i$. To do this, express the $(i, i+j)$ entry in this matrix for $j\ge 0$ (which is Cov$(X_i,X_{i+j})$) in terms of the $(i, i+j-2)$ and $(i,i+j-1)$ entries and solve the resulting linear recurrence relations. The solution, as a function of $i$ and $j$, will otherwise depend only on the known values $\alpha$ and $\beta$

(You might want to start by working out formulas for the diagonal entries Var$(X_i,X_i)$.)

4. The likelihood of the data $\mathrm{X} = (X_1, X_2, \ldots)$ is the probability density of a multivariate Normal distribution with parameters $(\mu,\Sigma)$ evaluated at $\mathrm{X}$, considered as a function of the unknown $\theta$, but depending explicitly on whatever values $\alpha$ and $\beta$ might be known to have.

5. The log likelihood is a quadratic function of $\theta$. Using a convenient formula for the locations of minima of quadratic functions (aka vertices of parabolas), write down the maximum likelihood estimate of $\theta$.

-
Thanks for that. I spent some time trying to work it out, but I'm still having trouble with step (3). I tried following your suggestions but, I can't seem to be able to write down the covariance matrix. Any help would be much appreciated! –  Just Jul 21 '14 at 23:02