# Log Likelihood for GLM

In the following code I perform a logistic regression on grouped data using glm and "by hand" using mle2. Why does the logLik function in R give me a log likelihood logLik(fit.glm)=-2.336 that is different than the one logLik(fit.ml)=-5.514 I get by hand?

library(bbmle)

#successes in first column, failures in second
Y <- matrix(c(1,2,4,3,2,0),3,2)

#predictor
X <- c(0,1,2)

#use glm
fit.glm <- glm(Y ~ X,family=binomial (link=logit))
summary(fit.glm)

#use mle2
invlogit <- function(x) { exp(x) / (1+exp(x))}
nloglike <- function(a,b) {
L <- 0
for (i in 1:n){
L <- L + sum(y[i,1]*log(invlogit(a+b*x[i])) +
y[i,2]*log(1-invlogit(a+b*x[i])))
}
return(-L)
}

fit.ml <- mle2(nloglike,
start=list(
a=-1.5,
b=2),
data=list(
x=X,
y=Y,
n=length(X)),
skip.hessian=FALSE)
summary(fit.ml)

#log likelihoods
logLik(fit.glm)
logLik(fit.ml)

y <- Y
x <- X
n <- length(x)
nloglike(coef(fit.glm)[1],coef(fit.glm)[2])
nloglike(coef(fit.ml)[1],coef(fit.ml)[2])

• A common reason for such differences is the fact that likelihood is only defined up to a multiplicative constant: "More precisely, then, a likelihood function is any representative from an equivalence class of functions, $\mathcal{L} \in \left\lbrace \alpha \; P_\theta: \alpha > 0 \right\rbrace, \,$ where the constant of proportionality $α > 0$ is not permitted to depend upon $θ$, and is required to be the same for all likelihood functions used in any one comparison." Log-likelihood may in turn be shifted by an arbitrary constant. ...(ctd) Commented Oct 6, 2013 at 20:31
• (ctd) ... That's not to say it's the explanation for this particular difference, but it's a common reason for differences between how different functions give different likelihoods. Commented Oct 6, 2013 at 20:33
• I have incorrectly assumed that the log likelihood was defined with the kernel of the pdf and was therefore unique for this problem.
– Tom
Commented Oct 6, 2013 at 20:59
• It's worth investigating, though, because sometimes the explanation is something else. Commented Oct 6, 2013 at 21:12

It appears that the logLik function in R calculates what is referred to in SAS as the "full likelihood function", which in this case includes the binomial coefficient. I did not include the binomial coefficient in the mle2 calculation because it has no impact on the parameter estimates. Once this constant is added to the log likelihood in the mle2 calculation, glm and mle2 agree.

• (+1) Thanks for following up and posting the resolution after you figured it out. Cheers. Commented Oct 8, 2013 at 10:34