23
$\begingroup$

Given an output from optim with a hessian matrix, how to calculate parameter confidence intervals using the hessian matrix?

fit<-optim(..., hessian=T)

hessian<-fit$hessian

I am mostly interested in the context of maximum likelihood analysis, but am curious to know if the method can be expanded beyond.

$\endgroup$
  • 2
    $\begingroup$ This question is too vague. What kind of confidence intervals? What sort of models are you interested on? Consider taking a look at the literature before posting 3 questions in a row. $\endgroup$ – user10525 Apr 24 '12 at 14:35
  • $\begingroup$ The method should be independent of the interval and model. $\endgroup$ – Etienne Low-Décarie Apr 24 '12 at 15:25
  • $\begingroup$ What function do you optimize ? $\endgroup$ – Stéphane Laurent Apr 24 '12 at 17:36
  • $\begingroup$ I was told I should be able to do this independent of the model I use. $\endgroup$ – Etienne Low-Décarie Apr 25 '12 at 20:38
  • $\begingroup$ Corey Chivers provided the answer. $\endgroup$ – Etienne Low-Décarie Apr 25 '12 at 21:12
32
$\begingroup$

If you are maximising a likelihood then the covariance matrix of the estimates is (asymptotically) the inverse of the negative of the Hessian. The standard errors are the square roots of the diagonal elements of the covariance (from elsewhere on the web!, from Prof. Thomas Lumley and Spencer Graves, Eng.).

For a 95% confidence interval

fit<-optim(pars,li_func,control=list("fnscale"=-1),hessian=TRUE,...)
fisher_info<-solve(-fit$hessian)
prop_sigma<-sqrt(diag(fisher_info))
prop_sigma<-diag(prop_sigma)
upper<-fit$par+1.96*prop_sigma
lower<-fit$par-1.96*prop_sigma
interval<-data.frame(value=fit$par, upper=upper, lower=lower)

Note that:

  • If you are maximizing the log(likelihood), then the NEGATIVE of the hessian is the "observed information" (such as here).
  • If you MINIMIZE a "deviance" = (-2)*log(likelihood), then the HALF of the hessian is the observed information.
  • In the unlikely event that you are maximizing the likelihood itself, you need to divide the negative of the hessian by the likelihood to get the observed information.

See this for further limitations due to optimization routine used.

$\endgroup$
  • $\begingroup$ Corey Chivers provided the answer. $\endgroup$ – Etienne Low-Décarie Apr 25 '12 at 21:12
  • 2
    $\begingroup$ (+1) The inverse of the negative hessian is an estimator of the asymptotic covariance matrix - I know this appears in your code but I think it's important to point out. $\endgroup$ – Macro Apr 26 '12 at 12:20
  • 1
    $\begingroup$ Excellent answer, should the upper and lower bounds read upper<-fit$par+1.96*(prop_sigma/sqrt(n)) lower<-fit$par-1.96*(prop_sigma/sqrt(n)) ? Thanks $\endgroup$ – forecaster Jun 16 '15 at 0:03
  • 3
    $\begingroup$ Why not delete line 4? $\endgroup$ – Jason Sep 12 '16 at 19:34
  • 2
    $\begingroup$ Is including the line prop_sigma<-diag(prop_sigma) an error? $\endgroup$ – Mark Miller Jan 21 '17 at 2:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.