6
$\begingroup$

When evaluating an estimator in a frequentist setting, using MSE and let say to compute the Bias of the estimator we compute the expectation of this estimator, are we supposing that the estimator has a probability distribution? doesn't this contradict the frequentist argument that a parameter is not a random variable?

$\endgroup$
  • 9
    $\begingroup$ An estimator is not a parameter, but a random variable. Basically, your estimate depends on the sample which is random, which makes your estimate a realisation of a random variable called estimator. Hope this helps. $\endgroup$ – ocram Apr 13 '12 at 4:47
  • 1
    $\begingroup$ Just to complement what @ocram said, this r.v that you formulate is also called a statistic. In general, a statistic is defined as a function of random variables which doesn't include any unkown variable. For example, think on the usual mean statistic (or estimator): $$\bar{X}=\sum_{i=1}^{N}\frac{X_i}{N}$$ If every r.v. is assumed to be taken from the same population (of "true" mean $\mu$), then, $$\mathbb{E}[\bar{X}]=\sum_{i=1}^{N}\frac{\mathbb{E}[X_i]}{N}$$ or $$\mathbb{E}[\bar{X}]=\sum_{i=1}^{N}\frac{\mu}{N}=\mu$$Therefore, $\bar{X}$ is an unbiased estimator of the mean... $\endgroup$ – Néstor Apr 13 '12 at 6:00
  • 1
    $\begingroup$ @ocram, I'd say your comment completely answers the question. Have you considered posting it as an answer? $\endgroup$ – Macro Apr 13 '12 at 12:22
  • 1
    $\begingroup$ @ocram: Please convert your comment to an answer so that I may upvote it. Cheers. $\endgroup$ – cardinal Apr 13 '12 at 12:28
  • 1
    $\begingroup$ @Nesp, if you look at even more serious books, they define a random variable as a mapping from the sample space to the set of real numbers. So if your sample space is reals^n for a univariate sample of size $n$, then indeed a function of the data provides an example of such a mapping. Moreover, any function of the data is a random variable. So the standard errors in regression are also random variables with their own distributions. The latter fact is probably understood by even fewer people than the fact that the slope estimate is a random variable. $\endgroup$ – StasK Apr 13 '12 at 16:45
10
$\begingroup$

An estimator is not a parameter, but a random variable. Basically, your estimate depends on the sample which is random, and this makes your estimate a realisation of a random variable called estimator.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.