Does this decision theory methodology have a name?

Question

Let's suppose that I have a quantity/parameter $\theta\in\Theta$ that I would like to make inferences about. I am able to draw data, say $X = (X_1,\ldots,X_n)$ I.I.D. from a distribution $P(x;\theta)$. I have a good handle on my data likelihood since it relies on trusted physics models.

I'm also willing to admit that $\theta$ is a random variable, but I have no good reason to assume that I know what a good prior distribution is - imagine that $\theta$ is very high dimensional and complex, and little is really known about its statistics - wild guesses have been tried but they all seem suspicious.

So, for lack of a good prior, I'd like to consider purely likelihood-based inference procedures, just to get started - i.e. I'll take a frequentist/likelihoodist approach to my inference - maximum likelihood estimators, confidence intervals, Fisher information, etc. So, suppose I've selected an estimation rule $\hat{\theta} = \delta(X)$.

Given this estimator, I can test its performance by choosing different priors $\pi(\theta)$, then evaluating something like a Bayes risk that depends on both the prior and the decision rule:

$$ E_\pi[L(\theta,\hat{\theta})] = \int_{\Theta}L(\theta,\hat{\theta})d\pi(\theta) $$Of course, as written, this Bayes risk is actually a random variable, since $\hat{\theta} = \delta(X)$ and $X$ is random; to deal with this I could either take the average over $X\vert \theta$ using the likelihood (fixed number of samples), or I could take a large-sample limit. At any rate, my definition of risk thus depends on both the prior and the decision rule, and the decision rule was formed independently of the prior used to test it. Thus on one hand I'd like to try forming an estimator independently of any prior, then I'd like to evaluate the resulting procedure using priors.

My question is: does this procedure make sense, and if so does it have a precedent/name in the statistics literature? I'm more of a math/physics person so I'm less familiar with the stats literature.

Answer 1

Using the quantity $$\mathbb{E}_\pi[L(\theta,\hat{\theta})] = \int_{\Theta}L(\theta,\hat{\theta})\text{d}\pi(\theta)$$ to evaluate the prior $\pi$ is not meaningful as such, but there may be interesting directions in this question:

1. The "best" prior is the Dirac mass at $\hat{\theta}$
2. The integral should be against $\pi(\theta|X)$ since $X$ has been observed
3. If one wants to integrate $X$ out the equality $$\int_\mathcal{X}\int_{\Theta}L(\theta,\hat{\theta}(x))\text{d}\pi(\theta|x)\text{d}m(x)=\int_{\Theta}\int_{\mathcal{X}}L(\theta,\hat{\theta}(x))\text{d}f(x|\theta)\text{d}\pi(\theta)$$ (Fubini's theorem) shows the order does not matter.
4. For the integrated risk above, the question gets more interesting.

For instance, in the normal mean N$(\theta,1)$ case, the integrated risks associated with conjugate priors on $\theta$ (and prior mean zero) look like \begin{align*}\mathbb{E}_m[\text{var}(\theta|X)]+\mathbb{E}_m[(\mathbb{E}(\theta|X)-X)^2]&=\dfrac{b}{1+b}+\mathbb{E}_m\left[\left(\dfrac{X}{1+b}\right)^2\right]\\&=\dfrac{b}{1+b}+\dfrac{1}{(1+b)^2}(1+b)\\ &=\dfrac{b}{1+b}+\dfrac{1}{(1+b)}=1\\\end{align*} which is obvious if $X$ is first integrated out since $$\mathbb{E}[(\theta-X)^2]=\mathbb{E}^\pi[\mathbb{E}^X[(\theta-X)^2]]=\mathbb{E}^\pi[1]=1$$ This result means these priors cannot be compared with respect to this measure.