# Approximating the error of maximum likelihood estimation

I have a log likelihood function of a model and I want to find $\mu$ and $\sigma^2$ which maximize the log likelihood. Since the log lik function is quite complex, I decided to use Nelder-Mead algorithm from scipy module. What I would like to do now is to estimate the errors on $\mu$ and $\sigma^2$ retrieved by Nelder-Mead algorithm.

How to do it without calculating the Hessian of the function?

EDIT: I found a way to approximate the Hessian (using numdifftools.core.Hessian) but I'm not sure how much can I rely on it.

EDIT 2: In some cases I manage to use the approximated Hessian (from numdifftools) but in other cases, it doesn't work. And it seems like there is a number precision problem. So I'm still looking for a solution with which I can efficiently asses the approximation error without using Hessian or an implementation of a Hessian approximation which works good in high number precision.

• Does the data follow some specific distribution? – HelloGoodbye Aug 22 '18 at 10:04
• Yes, it follows a Gaussian distribution. – Đorđe Relić Aug 22 '18 at 10:32
• Have you considered bootstrapping? – Denziloe Aug 22 '18 at 13:16
• @Denziloe I haven't. Mostly because of the reason it would not give me one error value for each parameter and also it would depend on the number of samples I choose, right? The thing is that I repeat the process of inferring mean and variance thousands of times (for many subsets of my given data set) and I would like to have a more convenient way of getting the error rather than sampling. – Đorđe Relić Aug 22 '18 at 14:04
• Sure, if optimisation is important then it makes sense to rule out bootstrapping. Choosing the number of samples isn't an issue though, you just need to choose "enough". The more the better but after a moderate number the answer should start to converge. And you'd get errors for each parameter. – Denziloe Aug 22 '18 at 14:07