Let $(x_1,y_1)(x_2,y_2)... (x_n,y_n)$ be independent pairs such that $y_i=\theta x_i+\epsilon_i$ where $x_i$ and $\epsilon_i$ are iid Normal (0,1) for $i=1,2..n$

It was previously computed that

$f(x,y)=\frac{1}{2\pi}e^{\frac{-1}{2}x^2-\frac12(y-\theta x)^2 }$

And that the MLE for $\theta$ is $\theta^*=\frac{\sum_{i=1}^nx_iy_i}{\sum_{i=1}^nx_i^2}$ which is unbiased for $\theta$

I need to know if this estimator achieves the CRLB.

Here is what I have so far:

For any estimator W(M) of $\theta$ using sample $M_i's$ of$M$ which are iid $i=1..n$, the CRLB is given by


So the numerator is 1 since $E(W(M))=\theta$.

For the denominator, I worked it out as $n(\theta^2(2-\theta^2))$.

Is this correct? It seems iffy coz it can have negative values.

Another problem is in computing the variance of the estimator.

$Var(\theta^*)=Var(\frac{\sum_{i=1}^nx_iy_i}{\sum_{i=1}^nx_i^2}$). How do I go about this?


2 Answers 2


The expression of your MLE isn't quite correct. The log-likelihood takes the form,

$$ \log(L(\textbf{x},\textbf{y};\theta) = constant - \frac{1}{2}\sum\limits_{i=1}^n(y_i - \theta x_i)^2 $$

Taking the derivative and setting equal to zero gives,

$$ \frac{d}{d\theta}\log(L(\textbf{x},\textbf{y};\theta) = \sum\limits_{i=1}^n(y_i - \theta x_i)x_i = 0 \Longrightarrow \hat{\theta} = \frac{\sum\limits_{i=1}^nx_iy_i}{\sum\limits_{i=1}^nx_i^2} $$

You can verify this gives a maximum using the second derivative.

  • $\begingroup$ You're right. I made a mistake evaluating the derivative. I will fix it now. $\endgroup$
    – user164144
    Jul 7, 2017 at 4:06

Ok. Here's my solution in determining if the estimator reaches the CRLB.

$Var(\theta^*)=Var(\frac{\sum_{i=1}^nx_iy_i}{\sum_{i=1}^nx_i^2})=Var(\frac{\sum_{i=1}^nx_i(\theta x_i+\epsilon_i)}{\sum_{i=1}^nx_i^2})$

$=Var(\frac{\sum_{i=1}^n(\theta x_i^2+x_i\epsilon_i)}{\sum_{i=1}^nx_i^2})$

$=Var(\frac{\theta\sum_{i=1}^n x_i^2}{\sum_{i=1}^nx_i^2}+\frac{\sum_{i=1}^nx_i\epsilon_i}{\sum_{i=1}^nx_i^2})$


At this point, I can argue that since $x_i's$ and $\epsilon_i's$ are iid $\sim N(0,1)$ then $Var(\theta^*)$ is independent of $\theta$ which means that it cannot attain the CRLB which is dependent on $\theta$. That is, there will always be a theta that can make the CRLB smaller than $Var(\theta^*)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.