How do I check for bias of an estimator?

Question

I need to check if an estimator $\hat\theta$ for the parameter $\theta$ is biased. Theory says I should compare the expected value of $\hat\theta$ versus the expected value of $\theta$.

I assume the expected value of an estimator is the "weighted average" of the estimator and its distribution: $E[\hat\theta] = \int_0^\inf \hat\theta f(\hat\theta) d\hat\theta$*. If I'm right, to compute $E[\hat\theta]$ I need to know how $\hat\theta$ is distributed.

For example:

$X$ is a random var with support $0 <= X <= \theta$ and pdf $f(x;\theta) = 3x^2 / \theta^3$. Check if $\delta(x)=(4\bar X)/3$ is biased.

*is it $E[\hat\theta] = \int_0^\inf \hat\theta f(\hat\theta) d\hat\theta$ or $E[\hat\theta] = \int_0^\inf \hat\theta f(x) dX$?

Answer 1

You seem to have some conceptual issues.

In the classical non-bayesian context (the fact that your are learning about bias, and your working example, suggest that this is your context) the parameter $\theta$ is ... a parameter, a number; which is perhaps unknown to us but which takes nonetheless some determined fixed value. In short: $\theta$ is not a random variable.

The estimator, instead, is (in general) a random variable. Because $\hat{\theta}=g(\{X\})$ where $g(\cdot)$ is some function and $\{X\}$ is a list of realizations ($X_1,X_2 \cdots.. X_n$) of a random variable. (Think for example, of the sample average $(X_1+X_2+\cdots + X_n)/n$) This is to say: in different "experiments" (trials) we'll get different values of the estimator $\hat{\theta}$ . But in all experiments the parameter $\theta$ will be the same.

That's why it makes sense to ask if $E(\hat{\theta})=\theta$ (because the left side is the expectation of a random variable, the right side is a constant). And, if the equation is valid (it might or not be, according to the estimator) the estimator is unbiased.

In your example, you're using $\hat{\theta} = \frac{X_1+X_2+ \cdots + X_n}{n}\frac{4}{3}$. The expectation of this is $E(\hat{\theta} )= \frac{n E(X)}{n} \frac{4}{3}$ Now, we need to compute $E(X)$ (it will be a function of $\theta$) and check if that gives $\theta$.