Skip to main content
Cross Validated

Return to Answer

light formatting
Source Link
gung - Reinstate Monica
gung - Reinstate Monica
  • 147.5k
  • 89
  • 406
  • 717

There's no statistical reason to prefer one to the other, besides conceptual clarity. Although the reported deviance values are different, these differences are completely due to the saturated model. So any comparison using relative deviance between models is unaffected, since the saturated model log-likelihood cancels.

I think it's useful to go through the explicit deviance calculation.

The deviance of a model is 2*(LL(Saturated Model) - LL(Model)). Suppose you have $i$ different cells, where $n_i$ is the number of observations in cell $i$, $p_i$ is the model prediction for all observations in cell $i$, and $y_{ij}$ is the observed value (0 or 1) for the $j$-th observation in cell $i$.

Long Form

The log likelihood of the (proposed or null) model is $$\sum_i\sum_j\left(\log(p_i)y_{ij} + \log(1 - p_i)(1 - y_{ij})\right)$$

and the log likelihood of the saturated model is $$\sum_i\sum_j \left(\log(y_{ij})y_{ij} + \log(1 - y_{ij})(1-y_{ij})\right).$$ This is equal to 0, because $y_{ij}$ is either 0 or 1. Note $\log(0)$ is undefined, but for convenience please read $0\log(0)$ as shorthand for $\lim_{x \to 0^+}x\log(x)$, which is 0.

Short form (weighted)

Note that a binomial distribution can't actually take non-integer values, but we can nonetheless calculate a "log likelihood" by using the fraction of observed successes in each cell as the response, and weighting each summand in the log-likelihood calculation by the number of observations in that cell.

$$\sum_in_i \left(log(p_i)\sum_jy_{ij}/n_i + \log(1 - p_i)(1 - \sum_j(y_{ij}/n_i)\right)$$$$\sum_in_i \left(\log(p_i)\sum_jy_{ij}/n_i + \log(1 - p_i)(1 - \sum_j(y_{ij}/n_i)\right)$$

This is exactly equal to the model deviance we calculated above, which you can see by pulling in the sum over $j$ in the long form equation as far as possible.

Meanwhile the saturated deviance is different. Since we no longer have 0-1 responses, even with one parameter per observation we can't get exactly 0. Instead the saturated model log-likelihood is

$$\sum_i n_i\left(\log(\sum_jy_{ij}/n_i)\sum_jy_{ij}/n_i + \log(1 - \sum_jy_{ij}/n_i)(1-\sum_jy_{ij}/n_i)\right).$$

In your example, you can verify that twice this amount is the difference between the reported null and residual deviance values for both models.

ni = dfShort$nReps
yavg = dfShort$mortalityP
sum.terms <-ni*(log(yavg)*yavg + log(1 - yavg)*(1 - yavg))
#Need# Need to handle NaN when yavg is exactly 0
sum.terms[1] <- log(1 - yavg[1])*(1 - yavg[1])

2*sum(sum.terms)
fitShortP$deviance - fitLong$deviance

There's no statistical reason to prefer one to the other, besides conceptual clarity. Although the reported deviance values are different, these differences are completely due to the saturated model. So any comparison using relative deviance between models is unaffected, since the saturated model log-likelihood cancels.

I think it's useful to go through the explicit deviance calculation.

The deviance of a model is 2*(LL(Saturated Model) - LL(Model)). Suppose you have $i$ different cells, where $n_i$ is the number of observations in cell $i$, $p_i$ is the model prediction for all observations in cell $i$, and $y_{ij}$ is the observed value (0 or 1) for the $j$-th observation in cell $i$.

Long Form

The log likelihood of the (proposed or null) model is $$\sum_i\sum_j\left(\log(p_i)y_{ij} + \log(1 - p_i)(1 - y_{ij})\right)$$

and the log likelihood of the saturated model is $$\sum_i\sum_j \left(\log(y_{ij})y_{ij} + \log(1 - y_{ij})(1-y_{ij})\right).$$ This is equal to 0, because $y_{ij}$ is either 0 or 1. Note $\log(0)$ is undefined, but for convenience please read $0\log(0)$ as shorthand for $\lim_{x \to 0^+}x\log(x)$, which is 0.

Short form (weighted)

Note that a binomial distribution can't actually take non-integer values, but we can nonetheless calculate a "log likelihood" by using the fraction of observed successes in each cell as the response, and weighting each summand in the log-likelihood calculation by the number of observations in that cell.

$$\sum_in_i \left(log(p_i)\sum_jy_{ij}/n_i + \log(1 - p_i)(1 - \sum_j(y_{ij}/n_i)\right)$$

This is exactly equal to the model deviance we calculated above, which you can see by pulling in the sum over $j$ in the long form equation as far as possible.

Meanwhile the saturated deviance is different. Since we no longer have 0-1 responses, even with one parameter per observation we can't get exactly 0. Instead the saturated model log-likelihood is

$$\sum_i n_i\left(\log(\sum_jy_{ij}/n_i)\sum_jy_{ij}/n_i + \log(1 - \sum_jy_{ij}/n_i)(1-\sum_jy_{ij}/n_i)\right).$$

In your example, you can verify that twice this amount is the difference between the reported null and residual deviance values for both models.

ni = dfShort$nReps
yavg = dfShort$mortalityP
sum.terms <-ni*(log(yavg)*yavg + log(1 - yavg)*(1 - yavg))
#Need to handle NaN when yavg is exactly 0
sum.terms[1] <- log(1 - yavg[1])*(1 - yavg[1])

2*sum(sum.terms)
fitShortP$deviance - fitLong$deviance

There's no statistical reason to prefer one to the other, besides conceptual clarity. Although the reported deviance values are different, these differences are completely due to the saturated model. So any comparison using relative deviance between models is unaffected, since the saturated model log-likelihood cancels.

I think it's useful to go through the explicit deviance calculation.

The deviance of a model is 2*(LL(Saturated Model) - LL(Model)). Suppose you have $i$ different cells, where $n_i$ is the number of observations in cell $i$, $p_i$ is the model prediction for all observations in cell $i$, and $y_{ij}$ is the observed value (0 or 1) for the $j$-th observation in cell $i$.

Long Form

The log likelihood of the (proposed or null) model is $$\sum_i\sum_j\left(\log(p_i)y_{ij} + \log(1 - p_i)(1 - y_{ij})\right)$$

and the log likelihood of the saturated model is $$\sum_i\sum_j \left(\log(y_{ij})y_{ij} + \log(1 - y_{ij})(1-y_{ij})\right).$$ This is equal to 0, because $y_{ij}$ is either 0 or 1. Note $\log(0)$ is undefined, but for convenience please read $0\log(0)$ as shorthand for $\lim_{x \to 0^+}x\log(x)$, which is 0.

Short form (weighted)

Note that a binomial distribution can't actually take non-integer values, but we can nonetheless calculate a "log likelihood" by using the fraction of observed successes in each cell as the response, and weighting each summand in the log-likelihood calculation by the number of observations in that cell.

$$\sum_in_i \left(\log(p_i)\sum_jy_{ij}/n_i + \log(1 - p_i)(1 - \sum_j(y_{ij}/n_i)\right)$$

This is exactly equal to the model deviance we calculated above, which you can see by pulling in the sum over $j$ in the long form equation as far as possible.

Meanwhile the saturated deviance is different. Since we no longer have 0-1 responses, even with one parameter per observation we can't get exactly 0. Instead the saturated model log-likelihood is

$$\sum_i n_i\left(\log(\sum_jy_{ij}/n_i)\sum_jy_{ij}/n_i + \log(1 - \sum_jy_{ij}/n_i)(1-\sum_jy_{ij}/n_i)\right).$$

In your example, you can verify that twice this amount is the difference between the reported null and residual deviance values for both models.

ni = dfShort$nReps
yavg = dfShort$mortalityP
sum.terms <-ni*(log(yavg)*yavg + log(1 - yavg)*(1 - yavg))
# Need to handle NaN when yavg is exactly 0
sum.terms[1] <- log(1 - yavg[1])*(1 - yavg[1])

2*sum(sum.terms)
fitShortP$deviance - fitLong$deviance
Bounty Ended with 50 reputation awarded by Richard Erickson
Note about 0log(0)
Source Link
Jonny Lomond
Jonny Lomond
  • 1.3k
  • 7
  • 12

There's no statistical reason to prefer one to the other, besides conceptual clarity. Although the reported deviance values are different, these differences are completely due to the saturated model. So any comparison using relative deviance between models is unaffected, since the saturated model log-likelihood cancels.

I think it's useful to go through the explicit deviance calculation.

The deviance of a model is 2*(LL(Saturated Model) - LL(Model)). Suppose you have $i$ different cells, where $n_i$ is the number of observations in cell $i$, $p_i$ is the model prediction for all observations in cell $i$, and $y_{ij}$ is the observed value (0 or 1) for the $j$-th observation in cell $i$.

Long Form

The log likelihood of the (proposed or null) model is $$\sum_i\sum_j\left(\log(p_i)y_{ij} + \log(1 - p_i)(1 - y_{ij})\right)$$

and the log likelihood of the saturated model is $$\sum_i\sum_j \left(\log(y_{ij})y_{ij} + \log(1 - y_{ij})(1-y_{ij})\right).$$ This is equal to 0, because $y_{ij}$ is either 0 or 1. Note $\log(0)$ is undefined, but for convenience please read $0\log(0)$ as shorthand for $\lim_{x \to 0^+}x\log(x)$, which is 0.

Short form (weighted)

Note that a binomial distribution can't actually take non-integer values, but we can nonetheless calculate a "log likelihood" by using the fraction of observed successes in each cell as the response, and weighting each summand in the log-likelihood calculation by the number of observations in that cell.

$$\sum_in_i \left(log(p_i)\sum_jy_{ij}/n_i + \log(1 - p_i)(1 - \sum_j(y_{ij}/n_i)\right)$$

This is exactly equal to the model deviance we calculated above, which you can see by pulling in the sum over $j$ in the long form equation as far as possible.

Meanwhile the saturated deviance is different. Since we no longer have 0-1 responses, even with one parameter per observation we can't get exactly 0. Instead the saturated model log-likelihood is

$$\sum_i n_i\left(\log(\sum_jy_{ij}/n_i)\sum_jy_{ij}/n_i + \log(1 - \sum_jy_{ij}/n_i)(1-\sum_jy_{ij}/n_i)\right).$$

In your example, you can verify that twice this amount is the difference between the reported null and residual deviance values for both models.

ni = dfShort$nReps
yavg = dfShort$mortalityP
sum.terms <-ni*(log(yavg)*yavg + log(1 - yavg)*(1 - yavg))
#Need to handle NaN when yavg is exactly 0
sum.terms[1] <- log(1 - yavg[1])*(1 - yavg[1])

2*sum(sum.terms)
fitShortP$deviance - fitLong$deviance

There's no statistical reason to prefer one to the other, besides conceptual clarity. Although the reported deviance values are different, these differences are completely due to the saturated model. So any comparison using relative deviance between models is unaffected, since the saturated model log-likelihood cancels.

I think it's useful to go through the explicit deviance calculation.

The deviance of a model is 2*(LL(Saturated Model) - LL(Model)). Suppose you have $i$ different cells, where $n_i$ is the number of observations in cell $i$, $p_i$ is the model prediction for all observations in cell $i$, and $y_{ij}$ is the observed value (0 or 1) for the $j$-th observation in cell $i$.

Long Form

The log likelihood of the (proposed or null) model is $$\sum_i\sum_j\left(\log(p_i)y_{ij} + \log(1 - p_i)(1 - y_{ij})\right)$$

and the log likelihood of the saturated model is $$\sum_i\sum_j \left(\log(y_{ij})y_{ij} + \log(1 - y_{ij})(1-y_{ij})\right).$$ This is equal to 0, because $y_{ij}$ is either 0 or 1.

Short form (weighted)

Note that a binomial distribution can't actually take non-integer values, but we can nonetheless calculate a "log likelihood" by using the fraction of observed successes in each cell as the response, and weighting each summand in the log-likelihood calculation by the number of observations in that cell.

$$\sum_in_i \left(log(p_i)\sum_jy_{ij}/n_i + \log(1 - p_i)(1 - \sum_j(y_{ij}/n_i)\right)$$

This is exactly equal to the model deviance we calculated above, which you can see by pulling in the sum over $j$ in the long form equation as far as possible.

Meanwhile the saturated deviance is different. Since we no longer have 0-1 responses, even with one parameter per observation we can't get exactly 0. Instead the saturated model log-likelihood is

$$\sum_i n_i\left(\log(\sum_jy_{ij}/n_i)\sum_jy_{ij}/n_i + \log(1 - \sum_jy_{ij}/n_i)(1-\sum_jy_{ij}/n_i)\right).$$

In your example, you can verify that twice this amount is the difference between the reported null and residual deviance values for both models.

ni = dfShort$nReps
yavg = dfShort$mortalityP
sum.terms <-ni*(log(yavg)*yavg + log(1 - yavg)*(1 - yavg))
#Need to handle NaN when yavg is exactly 0
sum.terms[1] <- log(1 - yavg[1])*(1 - yavg[1])

2*sum(sum.terms)
fitShortP$deviance - fitLong$deviance

There's no statistical reason to prefer one to the other, besides conceptual clarity. Although the reported deviance values are different, these differences are completely due to the saturated model. So any comparison using relative deviance between models is unaffected, since the saturated model log-likelihood cancels.

I think it's useful to go through the explicit deviance calculation.

The deviance of a model is 2*(LL(Saturated Model) - LL(Model)). Suppose you have $i$ different cells, where $n_i$ is the number of observations in cell $i$, $p_i$ is the model prediction for all observations in cell $i$, and $y_{ij}$ is the observed value (0 or 1) for the $j$-th observation in cell $i$.

Long Form

The log likelihood of the (proposed or null) model is $$\sum_i\sum_j\left(\log(p_i)y_{ij} + \log(1 - p_i)(1 - y_{ij})\right)$$

and the log likelihood of the saturated model is $$\sum_i\sum_j \left(\log(y_{ij})y_{ij} + \log(1 - y_{ij})(1-y_{ij})\right).$$ This is equal to 0, because $y_{ij}$ is either 0 or 1. Note $\log(0)$ is undefined, but for convenience please read $0\log(0)$ as shorthand for $\lim_{x \to 0^+}x\log(x)$, which is 0.

Short form (weighted)

Note that a binomial distribution can't actually take non-integer values, but we can nonetheless calculate a "log likelihood" by using the fraction of observed successes in each cell as the response, and weighting each summand in the log-likelihood calculation by the number of observations in that cell.

$$\sum_in_i \left(log(p_i)\sum_jy_{ij}/n_i + \log(1 - p_i)(1 - \sum_j(y_{ij}/n_i)\right)$$

This is exactly equal to the model deviance we calculated above, which you can see by pulling in the sum over $j$ in the long form equation as far as possible.

Meanwhile the saturated deviance is different. Since we no longer have 0-1 responses, even with one parameter per observation we can't get exactly 0. Instead the saturated model log-likelihood is

$$\sum_i n_i\left(\log(\sum_jy_{ij}/n_i)\sum_jy_{ij}/n_i + \log(1 - \sum_jy_{ij}/n_i)(1-\sum_jy_{ij}/n_i)\right).$$

In your example, you can verify that twice this amount is the difference between the reported null and residual deviance values for both models.

ni = dfShort$nReps
yavg = dfShort$mortalityP
sum.terms <-ni*(log(yavg)*yavg + log(1 - yavg)*(1 - yavg))
#Need to handle NaN when yavg is exactly 0
sum.terms[1] <- log(1 - yavg[1])*(1 - yavg[1])

2*sum(sum.terms)
fitShortP$deviance - fitLong$deviance
Source Link
Jonny Lomond
Jonny Lomond
  • 1.3k
  • 7
  • 12

There's no statistical reason to prefer one to the other, besides conceptual clarity. Although the reported deviance values are different, these differences are completely due to the saturated model. So any comparison using relative deviance between models is unaffected, since the saturated model log-likelihood cancels.

I think it's useful to go through the explicit deviance calculation.

The deviance of a model is 2*(LL(Saturated Model) - LL(Model)). Suppose you have $i$ different cells, where $n_i$ is the number of observations in cell $i$, $p_i$ is the model prediction for all observations in cell $i$, and $y_{ij}$ is the observed value (0 or 1) for the $j$-th observation in cell $i$.

Long Form

The log likelihood of the (proposed or null) model is $$\sum_i\sum_j\left(\log(p_i)y_{ij} + \log(1 - p_i)(1 - y_{ij})\right)$$

and the log likelihood of the saturated model is $$\sum_i\sum_j \left(\log(y_{ij})y_{ij} + \log(1 - y_{ij})(1-y_{ij})\right).$$ This is equal to 0, because $y_{ij}$ is either 0 or 1.

Short form (weighted)

Note that a binomial distribution can't actually take non-integer values, but we can nonetheless calculate a "log likelihood" by using the fraction of observed successes in each cell as the response, and weighting each summand in the log-likelihood calculation by the number of observations in that cell.

$$\sum_in_i \left(log(p_i)\sum_jy_{ij}/n_i + \log(1 - p_i)(1 - \sum_j(y_{ij}/n_i)\right)$$

This is exactly equal to the model deviance we calculated above, which you can see by pulling in the sum over $j$ in the long form equation as far as possible.

Meanwhile the saturated deviance is different. Since we no longer have 0-1 responses, even with one parameter per observation we can't get exactly 0. Instead the saturated model log-likelihood is

$$\sum_i n_i\left(\log(\sum_jy_{ij}/n_i)\sum_jy_{ij}/n_i + \log(1 - \sum_jy_{ij}/n_i)(1-\sum_jy_{ij}/n_i)\right).$$

In your example, you can verify that twice this amount is the difference between the reported null and residual deviance values for both models.

ni = dfShort$nReps
yavg = dfShort$mortalityP
sum.terms <-ni*(log(yavg)*yavg + log(1 - yavg)*(1 - yavg))
#Need to handle NaN when yavg is exactly 0
sum.terms[1] <- log(1 - yavg[1])*(1 - yavg[1])

2*sum(sum.terms)
fitShortP$deviance - fitLong$deviance