6
$\begingroup$

I am working on example 7.3.1 from the Second Edition of the book An Introduction to Generalized Linear Models in section 7.3 Dose response models. This example fits a simple logistic regression model on the following data:

enter image description here

This seems easy enough. However, I am having an issue with the Deviance Statistic calculated for this example. The following is my R code that will reproduce a Deviance Statistic $D=11.23$ just like this example in the book has.

#original data
#copied in by row
( df <-  data.frame( 
  Trial = 1:8,
  Dose = c(1.6907, 1.7242, 1.7552, 1.7842, 1.8113, 1.8369, 1.8610, 1.8839),
  Yes = c(6, 13, 18, 28, 52, 53, 61, 60),
  No = c(59, 60, 62, 56, 63, 59, 62, 60)- c(6, 13, 18, 28, 52, 53, 61, 60),
  Total = c(59, 60, 62, 56, 63, 59, 62, 60)
) )

#Logistic Regression Model
mle_beet <- glm(cbind(Yes, No)~Dose, family=binomial(logit), data=df)
mle_beet$deviance
##

Section 5.6.1 of this same book derives the Deviance Statistic for the Binomial Model to be:

$D = 2\sum^{N}_{i=1}y_{i}[ log_{e}(\frac{y_i}{\hat{y_i}})+(n_i - y_i)log_{e}(\frac{n_i - y_i}{n_i - \hat{y_i}}) ]$

However, looking closely at the given data, it can be seen that for the last row, the number of beetles killed is the same as the total number of beetles ( $n_{8}=y_{8}$ ). This means that the very last part in the sum for D is:

$ y_{8}log_{e}(\frac{y_8}{\hat{y_8}})+(n_8 - y_8)log_{e}(\frac{n_8 - y_8}{n_8 - \hat{y_8}}) = 60log_{e}(\frac{60}{\hat{y_8}})+(0)log_{e}(\frac{0}{n_8 - \hat{y_8}})$

In particular, this value contains:

$0log_{e}(0)=0(-\infty)=$ undefined

Here is the R code that agrees with this:

sum( 2*(df$Yes*(log(df$Yes/(mle_beet$fitted.values*df$Total))) + (df$Total-df$Yes)*
log((df$Total-df$Yes)/(df$Total-mle_beet$fitted.values*df$Total) ) ) )

My question is: What is the mathematical reasoning for computing the Deviance Statistic when $n_i=y_i$? What do the book and R do in the background to obtain $D=11.23$?

(Note that the book likely didn't use R to get this value, but the two agree)

Thank you!

EDIT: See the accepted answer and its comments for a great explanation.

If you happen to be computing the Deviance through the formula in R (you likely shouldn't since mle_beet$deviance shows this for you), you can replace -Inf or Nan in each vector that results from an individual operation. The following works for this example:

x <- df$Yes*(log(df$Yes/(mle_beet$fitted.values*df$Total))) 
x[is.na(x) | x==-Inf ] <- 0 #only in a case $n_i = y_i$ 
y <- (df$Total-df$Yes)*
    log((df$Total-df$Yes)/(df$Total-mle_beet$fitted.values*df$Total) ) ) 
y[is.na(y) | y==-Inf ] <- 0 #only in a case $n_i = y_i$ 

sum(x+y)*2 #the deviance
$\endgroup$
  • $\begingroup$ When calculating the binomial deviance we use the convention that $0 \log(0) = \lim_{x \downarrow 0} x \log(x) = 0$. $\endgroup$ – dsaxton Jul 13 '15 at 0:25
8
$\begingroup$

When calculating the binomial deviance we use the convention that $0 \log(0) = \lim_{x \downarrow 0} x \log(x) = 0$.

To understand why this convention makes sense here it's helpful for simplicity to think in terms of the Bernoulli likelihood function $\prod_{i=1}^{n} p_i^{y_i} (1 - p_i)^{1 - y_i}$ (the general binomial case will just be an extension of the main idea here), where $y_i$ is either zero or one.

Now when we talk about the saturated model (the likelihood of which is part of the deviance statistic) we mean the model where $\hat{p}_i = y_i$ for each $i$. But clearly the likelihood should equal one here (our model says that the data are certain to arise as they did), which means we should treat $0^0$ as itself being equal to one, and for the sake of consistency we should therefore treat the logarithm of $0^0$ as being equal to zero.

$\endgroup$
  • $\begingroup$ This makes some sense to me, but I still don't see, mathematically, why we can can go from $0log(0)$ to $\lim_{x\to{0^{+}}}xlog(x)$ besides that it's convention. $\endgroup$ – gbrlrz017 Jul 13 '15 at 2:19
  • $\begingroup$ I added some details to the post which hopefully clarify things somewhat. $\endgroup$ – dsaxton Jul 13 '15 at 2:52
  • $\begingroup$ Looking at it that way makes perfect sense! Here is a link to a related question on math.stackexchange that may help others understand this better, along with your answer, if they come across a similar problem. $\endgroup$ – gbrlrz017 Jul 13 '15 at 3:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.