2
$\begingroup$

Question 1

How do I interpret regression parameters in a logistic regression model if all my explanatory variables are not binary?

For example: I have two variables height (continuous) and color (takes on int values of 0, 1, and 2) where I use these 2 variables to model a binary response variable (ex. 1: at least one girlfriend and 0: no girlfriend). How would I interpret beta_height in terms of log-odds?

Question 2

Say again I want to model something similar, but now I want to change the variable color into multiple binary values (ie. factor(color) in R code). Now I have 4 regression parameters (beta_0, beta_height, beta_color1, beta_color2). How could I now interpret beta_height?

Question 3

How would I could I use a deviance test (residual deviance in R) to compare the model from question 1 and question 2 to see which model is "better"? Like how could I potentially set up the hypothesis testing here?

$\endgroup$
1
$\begingroup$

For Question 1, your binary logistic regression model would be expresed like this:

log(odds of having at least one girlfriend) = 
    beta_0 + beta_height * height + beta_color * color 

So you would interpret beta_height as follows:

Among subjects in the target population with the same value of color, each extra cm of height is associated with beta_height units change in the log odds of having at least one girlfriend.

For Question 2, your binary logistic regression model would be expresed like this:

log(odds of having at least one girlfriend) = 
   beta_0 + beta_height * height + 
    beta_color1 * color1 + beta_color2 * color2

Here, color1 is a dummy variable equal to 1 if color is equal to 1 and 0 if color is equal to 0 or 2. Furthermore, color2 is a dummy variable equal to 1 if color is equal to 2 and 0 if color is equal to 0 or 1. Note that the following holds:

When color1 = 0 & color2 = 0, color equals to 0.

When color1 = 1 & color2 = 0, color equals to 1.

When color1 = 0 & color2 = 1, color equals to 2. 

Because the model includes 2 dummy variables, you can think of it as a collection of 2 + 1 = 3 sub-models (one sub-model per color). The 3 sub-models are as follows:

Sub-model 1 [for color 0]

log(odds of having at least one girlfriend) = 
    beta_0 + beta_height * height

Sub-model 2 [for color 1]

log(odds of having at least one girlfriend) = 
   (beta_0 + beta_color1) + beta_height * height

Sub-model 3 [for color 2]

log(odds of having at least one girlfriend) = 
  (beta_0 + beta_color2) + beta_height * height

As you can see from the above, the original model assumes that the effect of height on the log odds of having at least one girlfriend is the same across colors. So you would interpret beta_height as follows:

Among subjects in the target population with the same value of color, each extra cm of height is associated with beta_height units change in the log odds of having at least one girlfriend.

Here, I assumed height was measured in cm. For beta_height > 0, the change would be an increase; for beta_height < 0, the change would be an decrease. Of course, beta_height is unknown and needs to be estimated from the data.

The Question 1 and Question 2 models assume a linear effect of height on the log odds of having at least one girlfriend given color. However, while the Question 1 model assumes a linear effect of color on these log odds given height, the Question 2 model allows the effect of color to be possibly non-linear.

One informal way to verify which of the two modelling options makes more sense for your data would be to fit the model for Question 2 and examine the estimated values of the coefficients beta_0, beta_color1 and beta_color2. If these coefficients increase at a linear rate, modelling the effect of colour as linear would make sense. In other words, if the estimated value of the difference beta_color1 - beta0 is roughly the same as the estimated value of the difference beta_color2 - beta_color1, then switch to using the Question 1 model.

You could also compare the AIC or BIC values of the two models to see which model has the lowest such value.

Because the models are nonnested, it makes sense to me to try and compare them (after fitting them to your data) using an R package like nonnest2, which was designed to implement Vuong’s (1989) theory of non-nested model comparison. Something like this:

model1 <- glm(outcome ~ height + color, 
              family = binomial(link="logit"), 
              data = yourdata)

model1 <- glm(outcome ~ height + factor(color), 
              family = binomial(link="logit"), 
              data = yourdata)

install.packages("nonnest2")

library("nonnest2")

vuongtest(model1, model2, nested=FALSE)

See https://cran.r-project.org/web/packages/nonnest2/vignettes/nonnest2.pdf.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.