1
$\begingroup$

In a maximum likelihood fit, one estimates the parameter with the mode of the likelihood $L$, and the variance of this estimator with the second derivative of $\log(L)$: $$ \bar\theta = \mathrm{Mode}[L] $$ $$ \mathrm{Var}[\bar\theta] \approx -\left(\frac{\partial^2}{\partial \theta^2}\log(L(\theta))\Big|_{\bar\theta}\right)^{-1} $$ From a bayesian point of view, it works because in the statistical limit the likelihood becomes normal and sharper than any regular enough prior, and this estimates are exact for a normal distribution. However, with not enough data, this estimate may become evidently biased.

I would like to raffinate this approximation using successive derivatives of $\log(L)$. My intent is to estimate at least the mean and variance of the posterior distribution (which is $L$ times the prior, then normalised). Using derivatives of $\log(p)$ has the advantage that it does not require normalisation: $$ \partial_x\log(N\cdot p(x))=\partial_x\big(\log(p(x))+N\big)=\partial_x\log(p(x)) $$

I could not find anything about this in the literature, but I'm not in the field so I hope either I just didn't use the right words, or there exists a better technique.

Trying to work out this, I got in trouble with the fact that odd derivatives in the expansion of $\log(p)$ give a divergence at infinity on one side when exponentiating the truncated series, so I could not even try to integrate the approximate pdf to directly compute the moments as a function of expansion coefficients.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.