# construct the maximum likelihood estimator

Let $a_{1},a_{2},a_{3}$ be independent with a normal(0,1) distribution. Define $X_{1},X_{2},X_{3}$ by $X_{1}=a_{1}$, $X_{2}=\theta X_{1}+a_{2}$ and $X_{3}=\theta X_{2}+a_{3}$ Find the MLE for $\theta$ .

My attempt: I get that $X_{1}$ has distribution $N(0,1)$, $X_{2}$ is $N(0,\theta^2+1)$ and $X_{3}$ is $N(0,\theta^4+\theta^2+1)$. Then I found the joint densisty and then the log likelihood function. Unfortunately, the expression is quite natsy and partial differentiation would involve simulataneous product, chain and quotient rule. This may be correct but can anyone confirm this is the right approach and the log likelihood function.

EDIT; The joint density of $X_{1},X_{2}, X_{3}$ I found to be $\frac{1}{\sqrt{8\pi^3 (\theta^2+1)(\theta^4+\theta^2+1)}}exp(\frac{-1}{2}(x^2+\frac{x^2}{\theta^2+1}+\frac{x^2}{\theta^4+\theta^2+1})$

• Are you referring to the joint density of $\{X_1, X_2, X_3\}$? Perhaps you should include it in the question. – Alecos Papadopoulos Apr 5 '14 at 17:41
• @AlecosPapadopoulos Now updated – user134724 Apr 5 '14 at 20:07
• It seems that you are treating the three $X$'s as independent random variables. Are they? – Alecos Papadopoulos Apr 5 '14 at 20:19
• Moreover, the joint density will be a tri-variate normal distribution. Is your task in the context of estimation from a multivariate distribution? – Alecos Papadopoulos Apr 5 '14 at 20:25
• @AlecosPapadopoulos My task is as stated. I figured the $X_{i}$ are independent since the $a_{i}$ are. Can you show me your attempt? – user134724 Apr 5 '14 at 20:47

The random variables $a_{1},a_{2},a_{3}$ are (standard) normal and independent. It follows that their joint probability distribution is tri-variate normal, $$\mathbf a = \left [\begin{matrix}\\ a_1\\ a_2\\ a_3\\ \end{matrix}\right] \sim \mathcal N_3 (\mathbf 0, \mathbf I_3)$$

where $\mathbf I_3$ is the $3 \times 3$ identity matrix. The random vector of the $X$'s is an afine (linear) transformation of $\mathbf a$, $\mathbf X = \mathbf \Theta \mathbf a$ so it, too, will have a trivariate normal distribution:

$$\mathbf X = \left [\begin{matrix}\\ X_1\\ X_2\\ X_3\\ \end{matrix}\right] = \left [\begin{matrix}\\ 1 & 0& 0\\ \theta & 1 & 0\\ \theta^2 & \theta & 1\\ \end{matrix}\right] \cdot \left [\begin{matrix}\\ a_1\\ a_2\\ a_3\\ \end{matrix}\right] \sim \mathcal N_3 (\mathbf 0, \mathbf \Sigma)$$

with

$$\Sigma = \Theta \Theta^T = \left [\begin{matrix}\\ 1 & \theta & \theta^2\\ \theta & \theta^2 +1 & \theta(\theta^2 +1)\\ \theta^2 & \theta(\theta^2 +1) & \theta^4+\theta^2+1\\ \end{matrix}\right]$$

Given just one realization $\mathbf x$ as the OP mentions in the comments, the density of the sample is

$$f_{\mathbf X} (\mathbf x) = (2\pi)^{-3/2}\cdot |\mathbf \Sigma|^{-1} \cdot \exp\left\{-\frac 12\mathbf x^T\mathbf \Sigma^{-1} \mathbf x\right\}$$

and the log-likelihood follows. Using the properties of the determinant you should quickly determine that $|\mathbf \Sigma| = 1$ and then find that the log-likelihood is just a 2nd-degree polynomial in $\theta$. The co-factor approach is the easiest one to calculate $\mathbf \Sigma ^{-1}$.