3
$\begingroup$

In this book, p.334 (348 for pdf) it says you can model a binomial regression in a few ways:

  1. response as an observed proportion, with weights. e.g.

    fit = glm(s/n ~ factor(group), weights=n, family="binomial")

  2. response is given as 2 columns array:

    fit = glm(cbind(Fissures, Turbines-Fissures) ~ Hours, family="binomial")

  3. response given as a factor (i.e. each row is a single Bernoulli trial):

    fit = glm(y ~ factor(group), family="binomial")

I ran options 1 and 3 on my dataset, and I get the exact same coefficients and p.values for them, BUT the Deviance and DF are different - for 1 I get that the residual deviance is too high, but for 3 it's actually very low.

Further in the chapter it is said that there are no Goodness-of-Fit for Binary Responses (i.e. for 3 I should ignore the residual-deviance), because:

"In this case the residual deviance and Pearson goodness-of fit statistics are determined entirely by the fitted values. This means that there is no concept of residual variability, and goodness-of-fit tests are not meaningful."

I don't understand why that is. Does anyone understand?

EDIT: here are the residual plots: enter image description here or against fitted values: enter image description here

Improve this question
$\endgroup$
3
  • $\begingroup$ For binary Y goodness of fit is assessed through modeling of specific departures from standard naiive assumptions such as linearity and additivity. This is done using methods such as interactions and relaxing linearity assumptions using e.g. regression splines. See hbiostat.org/rms for example. $\endgroup$
    – Frank Harrell
    Sep 1, 2020 at 11:40
  • $\begingroup$ @FrankHarrell what would you recommend as an alternative to Hosmer-Lemeshow? Further, if the observations in all cells are greater than five can you just use a chi-squared test? $\endgroup$
    – Single Malt
    Sep 1, 2020 at 17:00
  • $\begingroup$ If you have a single predictor that is categorical the logiistic model makes no assumptions at all other than independence of observations. If you only want a p-value (which is not asking very much) you can use the ordinary Pearson $\chi^2$ even with expected frequencies below 5 (it is a myth that you need 5). See the link I provided for general goodness-of-fit stuff. $\endgroup$
    – Frank Harrell
    Sep 3, 2020 at 11:28

1 Answer 1

Reset to default
2
$\begingroup$

I am assuming that in case 3, 'y' represents a rearrangement of the data, with each 'turbine' on its own line.

For case 3, the responses are binary, 0 or 1, so there is no residual deviance. This makes the Pearson goodness-of-fit meaningless. You can use the likelihood ratio test.

You can see this by doing a residual plot for each of the models, and comparing them.

Improve this answer
$\endgroup$
2
  • $\begingroup$ why is there no residual deviance? Can you show it mathematically? I plotted the 2 residual plots, and the 3rd one is indeed more weird, but they are similar in a way... $\endgroup$
    – Maverick Meerkat
    Sep 1, 2020 at 7:24
  • 1
    $\begingroup$ Residual deviance requires the log-likelihood of the saturated model. For the logistic model this is zero. So for logistic regression the residual is instead taken to be the observation value (zero or one) minus the predicted probability for that value. $\endgroup$
    – Single Malt
    Sep 1, 2020 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.