Let $a_{1},a_{2},a_{3}$ be independent with a normal(0,1) distribution. Define $X_{1},X_{2},X_{3}$ by $X_{1}=a_{1}$, $X_{2}=\theta X_{1}+a_{2}$ and $X_{3}=\theta X_{2}+a_{3}$ Find the MLE for $\theta$ .

My attempt: I get that $X_{1}$ has distribution $N(0,1)$, $X_{2}$ is $N(0,\theta^2+1)$ and $X_{3}$ is $N(0,\theta^4+\theta^2+1)$. Then I found the joint densisty and then the log likelihood function. Unfortunately, the expression is quite natsy and partial differentiation would involve simulataneous product, chain and quotient rule. This may be correct but can anyone confirm this is the right approach and the log likelihood function.

EDIT; The joint density of $X_{1},X_{2}, X_{3}$ I found to be $\frac{1}{\sqrt{8\pi^3 (\theta^2+1)(\theta^4+\theta^2+1)}}exp(\frac{-1}{2}(x^2+\frac{x^2}{\theta^2+1}+\frac{x^2}{\theta^4+\theta^2+1})$

  • $\begingroup$ Are you referring to the joint density of $\{X_1, X_2, X_3\}$? Perhaps you should include it in the question. $\endgroup$ Apr 5, 2014 at 17:41
  • $\begingroup$ @AlecosPapadopoulos Now updated $\endgroup$
    – user134724
    Apr 5, 2014 at 20:07
  • $\begingroup$ It seems that you are treating the three $X$'s as independent random variables. Are they? $\endgroup$ Apr 5, 2014 at 20:19
  • $\begingroup$ Moreover, the joint density will be a tri-variate normal distribution. Is your task in the context of estimation from a multivariate distribution? $\endgroup$ Apr 5, 2014 at 20:25
  • $\begingroup$ @AlecosPapadopoulos My task is as stated. I figured the $X_{i}$ are independent since the $a_{i}$ are. Can you show me your attempt? $\endgroup$
    – user134724
    Apr 5, 2014 at 20:47

1 Answer 1


The random variables $a_{1},a_{2},a_{3}$ are (standard) normal and independent. It follows that their joint probability distribution is tri-variate normal, $$ \mathbf a = \left [\begin{matrix}\\ a_1\\ a_2\\ a_3\\ \end{matrix}\right] \sim \mathcal N_3 (\mathbf 0, \mathbf I_3)$$

where $\mathbf I_3$ is the $3 \times 3$ identity matrix. The random vector of the $X$'s is an afine (linear) transformation of $\mathbf a$, $\mathbf X = \mathbf \Theta \mathbf a$ so it, too, will have a trivariate normal distribution:

$$ \mathbf X = \left [\begin{matrix}\\ X_1\\ X_2\\ X_3\\ \end{matrix}\right] = \left [\begin{matrix}\\ 1 & 0& 0\\ \theta & 1 & 0\\ \theta^2 & \theta & 1\\ \end{matrix}\right] \cdot \left [\begin{matrix}\\ a_1\\ a_2\\ a_3\\ \end{matrix}\right] \sim \mathcal N_3 (\mathbf 0, \mathbf \Sigma)$$


$$ \Sigma = \Theta \Theta^T = \left [\begin{matrix}\\ 1 & \theta & \theta^2\\ \theta & \theta^2 +1 & \theta(\theta^2 +1)\\ \theta^2 & \theta(\theta^2 +1) & \theta^4+\theta^2+1\\ \end{matrix}\right]$$

Given just one realization $\mathbf x$ as the OP mentions in the comments, the density of the sample is

$$f_{\mathbf X} (\mathbf x) = (2\pi)^{-3/2}\cdot |\mathbf \Sigma|^{-1} \cdot \exp\left\{-\frac 12\mathbf x^T\mathbf \Sigma^{-1} \mathbf x\right\}$$

and the log-likelihood follows. Using the properties of the determinant you should quickly determine that $|\mathbf \Sigma| = 1$ and then find that the log-likelihood is just a 2nd-degree polynomial in $\theta$. The co-factor approach is the easiest one to calculate $\mathbf \Sigma ^{-1}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.