Joint distribution of estimator and true parameter

Suppose we have probability distributions parameterized by $$\theta$$, and we have an estimator $$\hat{\Theta}$$ for $$\theta$$. Denote a fixed particular value of $$\hat{\Theta}$$ by the lower-case variable $$\hat{\theta}$$.

In frequentist inference, we look at the estimator $$\hat{\Theta}$$ as a random variable whose estimation properties depend on the true value $$\theta$$ (considered deterministic).

In what situations does it make sense to consider $$(\theta, \hat{\theta})$$ as a true joint distribution? In frequentist inference I believe we don't assume any prior on $$\theta$$, so that doesn't work.

• There is an issue with your understanding as you have communicated it. In frequentist point estimation, an estimator $\hat{\theta}$ is not a function of a parameter $\theta$. Rather, the estimator $\hat{\theta} = \hat{\theta}(X_1, \dots, X_n)$ is a function of the data $X_1, \dots X_n$. Because the data are random, $\hat{\theta}$ is a random variable. When the data is observed, the estimator, evaluated at observed values of the data, becomes an estimate, and so $\hat{\theta} = \hat{\theta}(x_1, \dots, x_n)$ is a number. Aug 7 at 0:40
• And the frequentist paradigm, as you’ve correctly stated, assumes that the parameter $\theta$ is a fixed unknown number. Importantly, that means it is not a random variable, and so, within this framework, it isn’t coherent to speak of a (non-degenerate) probability distribution on $\theta$. As far as I am aware, the same issue would befall an attempt to construct a joint distribution over an estimator $\hat{\theta}$ and parameter $\theta$. Aug 7 at 3:33
• @microhaus You’re right, function is the wrong word. But the behavior of $\hat{\Theta}$ (though still stochastic) certainly depends on the true value of the parameter theta. Aug 8 at 22:01