7
$\begingroup$

Let $X_i$ be an iid random variable having pdf $f(\mathbf{x}|\theta)$, where $E(X_i) = 6\theta^2$, and $\theta > 0$.

I have calculated an estimator for the parameter ($\theta$) of $f(\mathbf{x}|\theta)$ to be $\hat{\theta} = \sqrt{\bar{x}/6}$. To prove that this is an unbiased estimator, I should prove that $E(\hat{\theta}) = E\left(\sqrt{\bar{x}/6}\right)$. However, since $\hat{\theta}^2 = \bar{x}/6$, it would be much easier to show that $$\begin{align} E(\hat{\theta}^2) &= E(\bar{x}/6) \\ &=\frac{1}{6}E\left(\frac{\sum X_i}{n}\right)\\ &=\frac{1}{6n}\sum E(X_i) \\ &=\frac{1}{6n}n6\theta^2 \\&= \theta^2.\end{align}$$

Generally, proving $x^2 =4$ is not the same as proving $x=2$, since $x$ could also be $-2$. However, in this case $\theta>0$.

I have shown that $\hat{\theta}^2$ is unbiased, is this sufficient to show that $\hat{\theta}$ is unbiased?

$\endgroup$
  • 3
    $\begingroup$ Your title doesn't seem to make sense; it seems to be talking about estimating a random variable -- what you estimate is a parameter; Your last sentence says "I have shown that $θ^2$ is unbiased$ but parameters aren't biased or unbiased, ... estimators of parameters are. Please edit so your question is clear. $\endgroup$ – Glen_b -Reinstate Monica Apr 2 '17 at 1:52
  • $\begingroup$ Please see the help center on homework-style (routine textbook-type) questions (the discussion applies whether it's actually homework or not), then add the self-study tag as suggested there, and modify your question to follow the guidelines on asking such questions. In particular, you'd need to clearly identify what you've done to solve the problem yourself, and indicate the specific help you need at the point you struck difficulty. $\endgroup$ – Glen_b -Reinstate Monica Apr 2 '17 at 1:57
  • $\begingroup$ possible duplicate: stats.stackexchange.com/questions/271319/… $\endgroup$ – Taylor Apr 2 '17 at 2:08
  • $\begingroup$ @Taylor they're certainly related but the question here doesn't have the same answer as the question there. $\endgroup$ – Glen_b -Reinstate Monica Apr 2 '17 at 2:11
  • $\begingroup$ @Glen_b is right, the terminology is wrong here. But, I suspect you might be asking if an estimator is unbiased for $\theta^2$ then is the square root of that estimator unbiased for $\theta$. No, it is not. $\endgroup$ – gammer Apr 2 '17 at 2:11
11
$\begingroup$

Say $Q$ is unbiased for $\theta^2$, i.e. $E(Q) = \theta^2$, then because of Jensen's inequality,

$$\sqrt{ E(Q) } = \theta < E \left( \sqrt{Q} \right)$$

So $\sqrt{Q}$ is biased high, i.e. it will overestimate $\theta$ on average.

Note: This is a strict inequality (i.e. $<$ not $\leq$) because $Q$ is not a degenerate random variable and square root is not an affine transformation.

$\endgroup$
13
$\begingroup$

Note that for any estimator (with finite second moment) that $E(\widehat{\theta^2}) - E(\hat\theta)^2$ $=$ $\text{Var}(\hat\theta)\geq 0$ with equality only when $\text{Var}(\hat\theta)=0$ (which is easy to check doesn't hold).

Replace the first term on the LHS of that inequality by using your result for unbiasedness of $\widehat{\theta^2}$, and then by using the fact that $\theta$ and $\hat \theta$ are both positive, show $\hat \theta$ is biased, not unbiased as you supposed. (More generally, you could apply Jensen's inequality but it's not needed here)

Note that this proof doesn't relate to the particulars of your problem -- for a non-negative estimator of a non-negative parameter, if its square is unbiased for the square of the parameter, then the estimator must itself be biased unless the variance of the estimator is $0$.

$\endgroup$
  • 1
    $\begingroup$ +1 Just linking to the wikipedia article on Jensen's Inequality, since I found it super helpful when I was working through similar questions a few years ago $\endgroup$ – Rose Hartman Apr 2 '17 at 2:29
  • $\begingroup$ This is really clear and cool! $\endgroup$ – Zen Apr 5 '17 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.