3
$\begingroup$

I've been working on fitting a logistic regression model to a data set. I've attached the data set below. It includes data about the occurrence of snow, SnowBinary (=1 - if it snowed, 0 - if it did not snow) and the mean temperature, MeanTemp observed on that day.

In my GLM class, we were told that computing the deviance goodness of fit test for binomial data is not recommended. Instead, we should somehow bin the data before assessing fit.

I've conceived of two ways to do this. The first way is to round every MeanTemp observation to the nearest multiple of a number (I've chosen 5 for convenience) and then group the data. Here is how I might code it:

rounded.data = snow %>% 
              mutate(MeanTemp = round(MeanTemp*5)/5) %>% 
              group_by(MeanTemp) %>% 
              summarise(Snow = sum(SnowBinary), NoSnow = n() - Snow)

rounded.model = glm(cbind(Snow, NoSnow) ~ MeanTemp, data = rounded.data, family = binomial())
rounded.model %>% summary
#> 
#> Call:
#> glm(formula = cbind(Snow, NoSnow) ~ MeanTemp, family = binomial(), 
#>     data = rounded.data)
#> 
#> Deviance Residuals: 
#>     Min       1Q   Median       3Q      Max  
#> -2.1125  -0.8696  -0.4773   1.0710   1.9957  
#> 
#> Coefficients:
#>             Estimate Std. Error z value Pr(>|z|)    
#> (Intercept) -1.46084    0.43237  -3.379 0.000728 ***
#> MeanTemp    -0.13621    0.04585  -2.971 0.002969 ** 
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> (Dispersion parameter for binomial family taken to be 1)
#> 
#>     Null deviance: 69.888  on 48  degrees of freedom
#> Residual deviance: 59.314  on 47  degrees of freedom
#> AIC: 71.632
#> 
#> Number of Fisher Scoring iterations: 4

Another way suggested by a colleague, is to use cut to bin the data. His approach is shown below:

cut.data = snow %>% 
  mutate(MeanTemp = cut(MeanTemp,seq(-25,25,5))) %>% 
  group_by(MeanTemp) %>% 
  summarise(Snow = sum(SnowBinary), NoSnow = n() - Snow)

cut.model = glm(cbind(Snow, NoSnow) ~ MeanTemp, data = cut.data, family = binomial())
cut.model %>% summary
#> 
#> Call:
#> glm(formula = cbind(Snow, NoSnow) ~ MeanTemp, family = binomial(), 
#>     data = cut.data)
#> 
#> Deviance Residuals: 
#> [1]  0  0  0  0  0  0
#> 
#> Coefficients:
#>                     Estimate Std. Error z value Pr(>|z|)
#> (Intercept)           0.2877     0.7638   0.377    0.706
#> MeanTemp(-15,-10]    -0.6242     0.9624  -0.649    0.517
#> MeanTemp(-10,-5]      0.2231     0.9220   0.242    0.809
#> MeanTemp(-5,0]       -1.1632     0.9309  -1.249    0.212
#> MeanTemp(0,5]       -25.6412 53867.9146   0.000    1.000
#> MeanTemp(5,10]      -23.8538 79462.0050   0.000    1.000
#> 
#> (Dispersion parameter for binomial family taken to be 1)
#> 
#>     Null deviance: 1.8895e+01  on 5  degrees of freedom
#> Residual deviance: 3.7008e-10  on 0  degrees of freedom
#> AIC: 23.732
#> 
#> Number of Fisher Scoring iterations: 22

I am strongly suspicious that using cut is not preferable in this instance. For one, we must estimate more coefficients now (since cut returns a factor), and it creates problems of separation ( coefficient estimates for groups above 0 degrees have enormous standard errors).

I'm interested to hear what people here have to think about binning data in the aformention ways. When may cut be preferable to the method I suggest? Are both ways of assessing model fit bad? What is a better way of assessing model fit?

Data

snow = tibble::tribble(
  ~MeanTemp, ~SnowBinary,
        3.5,           0,
        3.7,           0,
        2.5,           0,
        4.9,           0,
          5,           0,
       -0.3,           0,
       -2.7,           0,
       -2.8,           0,
       -1.7,           0,
         -4,           1,
       -5.5,           0,
       -6.2,           1,
       -9.8,           1,
      -11.9,           0,
       -9.6,           0,
         -5,           1,
         -6,           0,
       -0.4,           0,
        3.4,           0,
       -0.9,           0,
         -2,           0,
       -2.6,           1,
       -1.7,           0,
       -6.9,           1,
       -9.9,           1,
      -15.9,           0,
        -16,           0,
      -14.9,           0,
      -12.6,           1,
      -15.8,           1,
      -18.4,           1,
      -15.6,           1,
      -13.4,           0,
      -13.4,           0,
      -14.2,           1,
      -18.3,           0,
      -19.9,           1,
      -13.8,           0,
       -1.1,           1,
       -7.4,           1,
       -3.6,           0,
        8.1,           0,
        0.4,           0,
        -12,           0,
      -10.8,           0,
       -9.6,           1,
       -9.2,           1,
      -10.8,           1,
       -7.9,           0,
       -2.6,           0,
        0.8,           0,
       -1.1,           0,
        4.5,           0,
        1.5,           0,
       -6.3,           0,
       -6.9,           1,
        0.1,           0,
        3.4,           0,
        0.7,           0,
       -5.9,           0,
      -10.6,           1,
       -4.9,           0,
       -4.6,           1,
      -11.1,           1,
       -6.6,           1,
       -4.8,           1
  )
$\endgroup$
  • 1
    $\begingroup$ Well the first approach modifies your data and is thus not really preferable, while binning also entails information loss. Why do you think you need to bin the continuous variable to assess model fit? Why don't you want to plot the deviance in function of the mean temperature and look for a trend? $\endgroup$ – Knarpie Feb 26 '18 at 10:27
  • $\begingroup$ The common approach is to bin the predicted probabilities, not the predictors. The problem with goodness of fit tests based on the deviance for binary logistic regression is that the likelihood of the saturated model is1. $\endgroup$ – jsk Jun 12 at 17:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.