In a maximum likelihood fit, one estimates the parameter with the mode of the likelihood $L$, and the variance of this estimator with the second derivative of $\log(L)$: $$ \bar\theta = \mathrm{Mode}[L] $$ $$ \mathrm{Var}[\bar\theta] \approx -\left(\frac{\partial^2}{\partial \theta^2}\log(L(\theta))\Big|_{\bar\theta}\right)^{-1} $$ From a bayesian point of view, it works because in the statistical limit the likelihood becomes normal and sharper than any regular enough prior, and this estimates are exact for a normal distribution. However, with not enough data, this estimate may become evidently biased.

I would like to raffinate this approximation using successive derivatives of $\log(L)$. My intent is to estimate at least the mean and variance of the posterior distribution (which is $L$ times the prior, then normalised). Using derivatives of $\log(p)$ has the advantage that it does not require normalisation: $$ \partial_x\log(N\cdot p(x))=\partial_x\big(\log(p(x))+N\big)=\partial_x\log(p(x)) $$

I could not find anything about this in the literature, but I'm not in the field so I hope either I just didn't use the right words, or there exists a better technique.

Trying to work out this, I got in trouble with the fact that odd derivatives in the expansion of $\log(p)$ give a divergence at infinity on one side when exponentiating the truncated series, so I could not even try to integrate the approximate pdf to directly compute the moments as a function of expansion coefficients.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.