Suppose we have probability distributions parameterized by $\theta$, and we have an estimator $\hat{\Theta}$ for $\theta$. Denote a fixed particular value of $\hat{\Theta}$ by the lower-case variable $\hat{\theta}$.

In frequentist inference, we look at the estimator $\hat{\Theta}$ as a random variable whose estimation properties depend on the true value $\theta$ (considered deterministic).

In what situations does it make sense to consider $(\theta, \hat{\theta})$ as a true joint distribution? In frequentist inference I believe we don't assume any prior on $\theta$, so that doesn't work.

  • 2
    $\begingroup$ There is an issue with your understanding as you have communicated it. In frequentist point estimation, an estimator $\hat{\theta}$ is not a function of a parameter $\theta$. Rather, the estimator $\hat{\theta} = \hat{\theta}(X_1, \dots, X_n)$ is a function of the data $X_1, \dots X_n$. Because the data are random, $\hat{\theta}$ is a random variable. When the data is observed, the estimator, evaluated at observed values of the data, becomes an estimate, and so $\hat{\theta} = \hat{\theta}(x_1, \dots, x_n)$ is a number. $\endgroup$
    – microhaus
    Aug 7 at 0:40
  • $\begingroup$ And the frequentist paradigm, as you’ve correctly stated, assumes that the parameter $\theta$ is a fixed unknown number. Importantly, that means it is not a random variable, and so, within this framework, it isn’t coherent to speak of a (non-degenerate) probability distribution on $\theta$. As far as I am aware, the same issue would befall an attempt to construct a joint distribution over an estimator $\hat{\theta}$ and parameter $\theta$. $\endgroup$
    – microhaus
    Aug 7 at 3:33
  • $\begingroup$ @microhaus You’re right, function is the wrong word. But the behavior of $\hat{\Theta}$ (though still stochastic) certainly depends on the true value of the parameter theta. $\endgroup$
    – Eric Auld
    Aug 8 at 22:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.