Skip to main content
Updated links that were "dead".
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157

The following explanation is not limited to logistic regression but applies equally in normal linear regression and other GLMs. Usually, R excludes one level of the categorical and the coefficients denote the difference of each class to this reference class (or sometimes called baseline class) (this is called dummy coding or treatment contrasts in R, see here and herehere for an excellent overview of the different contrast options). To see the current contrasts in R, type options("contrasts"). Normally, R orders the levels of the categorical variable alphabetically and takes the first as reference class. This is not always optimal and can be changed by typing (here, we would set the reference class to "c" in the new variable) new.variable <- relevel(old.variable, ref="c"). For each coefficient of every level of the categorical variable, a Wald test is performed to test whether the pairwise difference between the coefficient of the reference class and the other class is different from zero or not. This is what the $z$ and $p$-values in the regression table are. If only one categorical class is significant, this does not imply that the whole variable is meaningless and should be removed from the model. You can check the overall effect of the variable by performing a likelihood ratio test: fit two models, one with and one without the variable and type anova(model1, model2, test="LRT") in R (see example below). Here is an example:

The following explanation is not limited to logistic regression but applies equally in normal linear regression and other GLMs. Usually, R excludes one level of the categorical and the coefficients denote the difference of each class to this reference class (or sometimes called baseline class) (this is called dummy coding or treatment contrasts in R, see here and here for an excellent overview of the different contrast options). To see the current contrasts in R, type options("contrasts"). Normally, R orders the levels of the categorical variable alphabetically and takes the first as reference class. This is not always optimal and can be changed by typing (here, we would set the reference class to "c" in the new variable) new.variable <- relevel(old.variable, ref="c"). For each coefficient of every level of the categorical variable, a Wald test is performed to test whether the pairwise difference between the coefficient of the reference class and the other class is different from zero or not. This is what the $z$ and $p$-values in the regression table are. If only one categorical class is significant, this does not imply that the whole variable is meaningless and should be removed from the model. You can check the overall effect of the variable by performing a likelihood ratio test: fit two models, one with and one without the variable and type anova(model1, model2, test="LRT") in R (see example below). Here is an example:

The following explanation is not limited to logistic regression but applies equally in normal linear regression and other GLMs. Usually, R excludes one level of the categorical and the coefficients denote the difference of each class to this reference class (or sometimes called baseline class) (this is called dummy coding or treatment contrasts in R, see here for an excellent overview of the different contrast options). To see the current contrasts in R, type options("contrasts"). Normally, R orders the levels of the categorical variable alphabetically and takes the first as reference class. This is not always optimal and can be changed by typing (here, we would set the reference class to "c" in the new variable) new.variable <- relevel(old.variable, ref="c"). For each coefficient of every level of the categorical variable, a Wald test is performed to test whether the pairwise difference between the coefficient of the reference class and the other class is different from zero or not. This is what the $z$ and $p$-values in the regression table are. If only one categorical class is significant, this does not imply that the whole variable is meaningless and should be removed from the model. You can check the overall effect of the variable by performing a likelihood ratio test: fit two models, one with and one without the variable and type anova(model1, model2, test="LRT") in R (see example below). Here is an example:

Updated link to data, the old was no longer working.
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157
mydata <- read.csv("http"https://wwwstats.atsidre.ucla.edu/stat/data/binary.csv")

mydata$rank <- factor(mydata$rank)

my.mod <- glm(admit ~ gre + gpa + rank, data = mydata, family = "binomial")

summary(my.mod)

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept) -3.989979   1.139951  -3.500 0.000465 ***
gre          0.002264   0.001094   2.070 0.038465 *  
gpa          0.804038   0.331819   2.423 0.015388 *  
rank2       -0.675443   0.316490  -2.134 0.032829 *  
rank3       -1.340204   0.345306  -3.881 0.000104 ***
rank4       -1.551464   0.417832  -3.713 0.000205 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
mydata <- read.csv("http://www.ats.ucla.edu/stat/data/binary.csv")

mydata$rank <- factor(mydata$rank)

my.mod <- glm(admit ~ gre + gpa + rank, data = mydata, family = "binomial")

summary(my.mod)

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept) -3.989979   1.139951  -3.500 0.000465 ***
gre          0.002264   0.001094   2.070 0.038465 *  
gpa          0.804038   0.331819   2.423 0.015388 *  
rank2       -0.675443   0.316490  -2.134 0.032829 *  
rank3       -1.340204   0.345306  -3.881 0.000104 ***
rank4       -1.551464   0.417832  -3.713 0.000205 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
mydata <- read.csv("https://stats.idre.ucla.edu/stat/data/binary.csv")

mydata$rank <- factor(mydata$rank)

my.mod <- glm(admit ~ gre + gpa + rank, data = mydata, family = "binomial")

summary(my.mod)

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept) -3.989979   1.139951  -3.500 0.000465 ***
gre          0.002264   0.001094   2.070 0.038465 *  
gpa          0.804038   0.331819   2.423 0.015388 *  
rank2       -0.675443   0.316490  -2.134 0.032829 *  
rank3       -1.340204   0.345306  -3.881 0.000104 ***
rank4       -1.551464   0.417832  -3.713 0.000205 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
Source Link

The following explanation is not limited to logistic regression but applies equally in normal linear regression and other GLMs. Usually, R excludes one level of the categorical and the coefficients denote the difference of each class to this reference class (or sometimes called baseline class) (this is called dummy coding or treatment contrasts in R, see here and here for an excellent overview of the different contrast options). To see the current contrasts in R, type options("contrasts"). Normally, R orders the levels of the categorical variable alphabetically and takes the first as reference class. This is not always optimal and can be changed by typing (here, we would set the reference class to "c" in the new variable) new.variable <- relevel(old.variable, ref="c"). For each coefficient of every level of the categorical variable, a Wald testWald test is performed to test whether the pairwise difference between the coefficient of the reference class and the other class is different from zero or not. This is what the $z$ and $p$-values in the regression table are. If only one categorical class is significant, this does not imply that the whole variable is meaningless and should be removed from the model. You can check the overall effect of the variable by performing a likelihood ratio testlikelihood ratio test: fit two models, one with and one without the variable and type anova(model1, model2, test="LRT") in R (see example below). Here is an example:

This postThis post is also very interesting.

The following explanation is not limited to logistic regression but applies equally in normal linear regression and other GLMs. Usually, R excludes one level of the categorical and the coefficients denote the difference of each class to this reference class (or sometimes called baseline class) (this is called dummy coding or treatment contrasts in R, see here and here for an excellent overview of the different contrast options). To see the current contrasts in R, type options("contrasts"). Normally, R orders the levels of the categorical variable alphabetically and takes the first as reference class. This is not always optimal and can be changed by typing (here, we would set the reference class to "c" in the new variable) new.variable <- relevel(old.variable, ref="c"). For each coefficient of every level of the categorical variable, a Wald test is performed to test whether the pairwise difference between the coefficient of the reference class and the other class is different from zero or not. This is what the $z$ and $p$-values in the regression table are. If only one categorical class is significant, this does not imply that the whole variable is meaningless and should be removed from the model. You can check the overall effect of the variable by performing a likelihood ratio test: fit two models, one with and one without the variable and type anova(model1, model2, test="LRT") in R (see example below). Here is an example:

This post is also very interesting.

The following explanation is not limited to logistic regression but applies equally in normal linear regression and other GLMs. Usually, R excludes one level of the categorical and the coefficients denote the difference of each class to this reference class (or sometimes called baseline class) (this is called dummy coding or treatment contrasts in R, see here and here for an excellent overview of the different contrast options). To see the current contrasts in R, type options("contrasts"). Normally, R orders the levels of the categorical variable alphabetically and takes the first as reference class. This is not always optimal and can be changed by typing (here, we would set the reference class to "c" in the new variable) new.variable <- relevel(old.variable, ref="c"). For each coefficient of every level of the categorical variable, a Wald test is performed to test whether the pairwise difference between the coefficient of the reference class and the other class is different from zero or not. This is what the $z$ and $p$-values in the regression table are. If only one categorical class is significant, this does not imply that the whole variable is meaningless and should be removed from the model. You can check the overall effect of the variable by performing a likelihood ratio test: fit two models, one with and one without the variable and type anova(model1, model2, test="LRT") in R (see example below). Here is an example:

This post is also very interesting.

added 132 characters in body
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157
Loading
added 146 characters in body
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157
Loading
added 146 characters in body
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157
Loading
added 146 characters in body
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157
Loading
added 88 characters in body
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157
Loading
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157
Loading