The ratio of a maximized likelihood and a marginal likelihood

Question

I stumbled upon the following quantity and I'm wondering if anyone knows of anywhere it has appeared in the stats literature previously. Here's the setting:

Suppose you will observe data $y\in \mathcal{Y}$ which is distributed according to the density $p(y\,\vert\,\theta)$ for some parameter $\theta$ having a prior density $p(\theta).$ The marginal likelihood of $y$ is $p(y) = \int p(y\,\vert\,\theta)\,p(\theta)\,\text{d}\theta.$ Let $\theta_\text{ml}(y)$ be the maximum likelihood estimator of $\theta$ based on $y.$

The quantity of interest is $$c=\sup_{y\in \mathcal{Y}} \frac{p(y\,\vert\,\theta_\text{ml}(y))}{p(y)}.$$

I know that $c > 1,$ but I don't know much more than that. It certainly appears impossible to compute except perhaps in the simplest settings. I can't say immediately whether or when it is finite. I'm sure that it's also hard to bound above, which is what I'd be most interested in doing.

Answer 1

The quantity $p(y|\theta_\text{ml}(y))$ is the marginal likelihood for the most favourable prior, which is not a true prior since it depends on $y$. It simply achieves the upper bound. Taking the supremum over all $y$'s does not make sense from a Bayesian perspective, since it should be conditional on $y$.

Now if one considers the Normal-Normal case, with prior $\theta\sim\mathcal{N}(0,1)$ and observation $X\sim\mathcal{N}(\theta,1)$, we have $$p(y|\theta_\text{ml}(y))=\frac{1}{\sqrt{2\pi}}\qquad \text{and}\qquad m(y)=\frac{1}{\sqrt{4\pi}}\exp\{-y^2/4\}$$ Hence $$\sup_{y\in \mathcal{Y}} \frac{p(y\,\vert\,\theta_\text{ml}(y))}{m(y)}=\infty$$ which is not very surprising in that $p(y|\theta_\text{ml}(y))$ will correspond to an overfit of the model, while $m(y)$ includes a penalty à la Schwarz (BIC). This may also connects with Lindley's paradox.