Suppose we have a parameter $\theta$ that we want to estimate. We sample an observation (random variable) $X$ from a known distribution $D_{X|\theta}$. Then, we can compute an estimator $\hat\theta(X)$ as a function of $X$.
In normal parameter estimation, $\theta$ is unknown but fixed. This essentially means that we don't make the assumption that we know anything at all about $\theta$. However, it seems reasonable to me that we might have some knowledge about what $\theta$ might look like. To represent this, let $\theta$ be a random variable in some fixed known distribution $D_\theta$, instead of being a fixed unknown value.
I think that the existence of $D_\theta$ and $D_{X|\theta}$ also implies that ($\theta$, $X$) have some fixed known joint distribution $D_{\theta,X}$ (if that's not the case, let's just assume that the distributions are well-behaved enough such that this joint distribution exists). Therefore, there exists a conditional distribution $D_{\theta | X}$ in addition to the usual $D_{X | \theta}$.
With this formulation, $\hat\theta$ is an unbiased estimator (in the usual sense) of $\theta$ iff $E[\hat\theta(X)|\theta] = \theta$. That is, for any fixed true value, the expectation of the estimator is equal to the true value. There have been a lot of things written about how to find unbiased estimators.
However, what I want is an estimator $\hat \theta'(X)$ such that it’s “unbiased in the other way around”. Formally, $E[\theta | X] = \hat\theta'(X)$, that is, for any fixed observation, and therefore fixed estimated value, the expected value of the true value is equal to that fixed estimated value. How would one go about formulating such an estimator? Is there a name for this kind of estimator?
