Confusion about asymptotic distribution of the MLE and of the MAP

Question

It's well known that the MLE $\hat{\theta}$ maximizes $f(y\mid\theta)$ and under regularity conditions has asymptotic distribution $$N\left(\theta, \frac{I(\theta)}{J^2(\theta)} \right)$$ where $I(\theta)=Var(E[\partial\ell'(y_i,\theta)])$ and $J(\theta)=-E[\partial^2\ell(y_i,\theta)]$(for simplicity I treat the single parameter case).

the MAP estimate gives $\theta$ a prior density and seeks to maximize $f(\theta\mid y)$. It can also be shown that $f(\theta\mid y)$ in the limit converges to $$N(\hat{\theta}, -\partial^2\ell(\theta\mid y))|_{\theta=\hat{\theta}})$$ where $\hat{\theta}$ denotes the mean of the posterior distribution, ie the MAP. Can I use this result to attain the asymptotic distribution of the MAP, similarly to the MLE? I thought it might have something to do with conjugate distributions (about which I know little)? It would already be very helpful if this were true in a special case, ie the normal case.

Answer 1

Let $g(\theta)$ be the prior distributionfor $\theta$. Let $\mathcal L_n$ be the log-likelhood of the sample,

$$\mathcal L_n = \sum_{i=1}^n \ln f(y_i \mid \theta)$$

The MLE maximizes just $\mathcal L_n$, or equivalently $(1/n)\mathcal L_n$.

The MAP maximizes (by considering the monotonic logarithmic transformation) $\ln g(\theta) + \mathcal L_n$, so the f.o.c. is

$$\hat \theta_{MAP}: \frac{g'(\hat \theta_{MAP})}{g(\hat \theta_{MAP})}+\frac{\partial \mathcal L_n(\hat \theta_{MAP})}{\partial \theta} = 0.$$

But, then we also have

$$\hat \theta_{MAP}: \frac 1n \left[\frac{g'(\hat \theta_{MAP})}{g(\hat \theta_{MAP})}+\frac{\partial \mathcal L_n(\hat \theta_{MAP})}{\partial \theta} \right]= 0$$

$$\implies \hat \theta_{MAP}: \frac 1n \frac{g'(\hat \theta_{MAP})}{g(\hat \theta_{MAP})}+\frac 1n\frac{\partial \mathcal L_n(\hat \theta_{MAP})}{\partial \theta} = 0.$$

But this implies that as $n \to \infty$ the first term goes to zero, because it is not some sum that increases as sample size increases, and so the MAP will become stochastically equivalent to the MLE. Intuitively, as we increase the size of the sample, the prior information embodied in $g(\theta)$ becomes negligible, compared to the information contained in the ever-increasing sample.