The difference of Standard Error between glm(y~x, family=poisson(link=identity)) and optim() in R I'm executing the following program glm(y~x, family=poisson(link=identity)). I can't understand the difference of SE from glm(y~x, family=poisson(link=identity)) and optim(). Please give me some advice.
optim()
x<-c(1,2,3,4)
y<-c(2,3,5,4)
f<-function(arg){
  a=arg[1]
  b=arg[2]
  mu=a+b*x
  -sum(y*log(mu)-mu-log(factorial(y)))
}
(reso<-optim(c(1,1),f,hessian=TRUE))
#$par
#[1] 1.2785717 0.8886162
#$hessian
#[1,] 1.239716 2.717462
#[2,] 2.717462 7.343171
sqrt(diag(solve(reso$hessian)))#SE from optim()
#[1] 2.0669196 0.8492641

glm()
resg<-glm(y~x,family=poisson(link=identity))
summary(resg)
#            Estimate Std. Error z value Pr(>|z|)
#(Intercept)   1.2784     1.9766   0.647    0.518
#x             0.8887     0.8141   1.092    0.275
diag(sqrt(vcov(resg)))#SE from glm(poisson, identity)
#(Intercept)           x 
#   1.976575    0.814139

 A: In statistical likelihood theory, minus the second derivative of the log-likelihood function is called the observed information. We might write this as
$$
I = -\ddot \ell(y; \theta)
$$
where the dots indicate differentiation with respect to $\theta$. The expected value of the observed information
$$
{\cal I} = E(I)
$$
is called Fisher information or expected information.
Observed and expected information are asymptotically equivalent (by the law of large numbers) under the same regularity conditions that guarantee that the maximum likelihood estimators are consistent. This implies that the observed and expected information will usually be close, in relative as well as absolute terms, when the standard errors are small.
In glm theory, Fisher information is preferred over observed information because it (1) has a much simpler analytic form, (2) is guaranteed to be positive definite (which observed information is not) and (3) is the same as the Cramer-Rao Lower Bound for the variance of unbiased estimators.
If the glm model has a canonical link, then the distinction is less important because observed and expected information are in that case identical when computed at the maximum likelihood estimator of $\theta$.
If you use optim to maximize the log-likelihood of a glm model, then the maximum likelihood estimates returned by glm and optim will be the same apart from rounding errors if both algorithms are run to convergence.
The standard errors from glm however will generally differ from those from optim because

*

*glm returns Fisher information whereas optim computes observed information and


*glm uses an exact analytic formula for Fisher information whereas optim approximates the Hessian numerically from second differences of the log-likelihood.
Had you had used a log-link instead of identity link for your example, then item 1 would no longer cause any difference because the log-link is canonical and observed and expected information would then become identical at convergence.
Note on terminology
Some authors use "observed Fisher information" as a synonym for "observed information" and "expected Fisher information" as a synonym for "Fisher information". I think this terminology probably originates from Efron & Hinkley (1978):
B. Efron and D.V. Hinkley (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information. Biometrika 65(3), 457–483.
