Skip to main content
corrected glm family argument from poisson to binomial
Source Link
Stephan Kolassa
  • 130.8k
  • 22
  • 265
  • 499

You shouldn't be rewriting your categorical data into dummies yourself. Just feed your categorical data into glm as-is; R will internally turn it into dummies. Then stepAIC() and similar methods can work correctly and either keep or remove the entire predictor, not only single levels.

Alternatively, you can look at whether collapsing different levels of your categorical variable improves matters, either using AIC or with Likelihood Ratio or Wald tests. As an example, let's create some toy data:

set.seed(1)
foo <- data.frame(dv=sample(x=c(0,1),size=100,replace=TRUE),
  iv=sample(LETTERS[1:5],size=100,replace=TRUE))
model <- glm(dv~iv,family="poisson"family="binomial",data=foo)
AIC(model)
[1] 140.7574

Next, we collapse a couple of levels of foo$iv into single levels of a new predictor and fit a model with the new predictor:

foo$new.iv <- NA
foo$new.iv[foo$iv %in% c("A","B")] <- "a"
foo$new.iv[foo$iv %in% c("C","D")] <- "b"
foo$new.iv[foo$iv == "E"] <- "c"
AIC(update(model,.~new.iv))
[1] 137.8161

AIC is lower with collapsed factor levels. This is not surprising, given that there is no relationship between foo$iv and the dependent variable, so reducing the degrees of freedom should always reduce AIC.

Of course, you should not simply test all possible combinations of factor levels for collapsing; that would be data dredging and even using AIC would invalidate subsequent inferences if you do not account for this process. Instead, let theory on how the data were generated guide you. And report what you did so consumers of your analysis can get a feeling how much iterative model fitting was involved.

You shouldn't be rewriting your categorical data into dummies yourself. Just feed your categorical data into glm as-is; R will internally turn it into dummies. Then stepAIC() and similar methods can work correctly and either keep or remove the entire predictor, not only single levels.

Alternatively, you can look at whether collapsing different levels of your categorical variable improves matters, either using AIC or with Likelihood Ratio or Wald tests. As an example, let's create some toy data:

set.seed(1)
foo <- data.frame(dv=sample(x=c(0,1),size=100,replace=TRUE),
  iv=sample(LETTERS[1:5],size=100,replace=TRUE))
model <- glm(dv~iv,family="poisson",data=foo)
AIC(model)

Next, we collapse a couple of levels of foo$iv into single levels of a new predictor and fit a model with the new predictor:

foo$new.iv <- NA
foo$new.iv[foo$iv %in% c("A","B")] <- "a"
foo$new.iv[foo$iv %in% c("C","D")] <- "b"
foo$new.iv[foo$iv == "E"] <- "c"
AIC(update(model,.~new.iv))

AIC is lower with collapsed factor levels. This is not surprising, given that there is no relationship between foo$iv and the dependent variable, so reducing the degrees of freedom should always reduce AIC.

Of course, you should not simply test all possible combinations of factor levels for collapsing; that would be data dredging and even using AIC would invalidate subsequent inferences if you do not account for this process. Instead, let theory on how the data were generated guide you. And report what you did so consumers of your analysis can get a feeling how much iterative model fitting was involved.

You shouldn't be rewriting your categorical data into dummies yourself. Just feed your categorical data into glm as-is; R will internally turn it into dummies. Then stepAIC() and similar methods can work correctly and either keep or remove the entire predictor, not only single levels.

Alternatively, you can look at whether collapsing different levels of your categorical variable improves matters, either using AIC or with Likelihood Ratio or Wald tests. As an example, let's create some toy data:

set.seed(1)
foo <- data.frame(dv=sample(x=c(0,1),size=100,replace=TRUE),
  iv=sample(LETTERS[1:5],size=100,replace=TRUE))
model <- glm(dv~iv,family="binomial",data=foo)
AIC(model)
[1] 140.7574

Next, we collapse a couple of levels of foo$iv into single levels of a new predictor and fit a model with the new predictor:

foo$new.iv <- NA
foo$new.iv[foo$iv %in% c("A","B")] <- "a"
foo$new.iv[foo$iv %in% c("C","D")] <- "b"
foo$new.iv[foo$iv == "E"] <- "c"
AIC(update(model,.~new.iv))
[1] 137.8161

AIC is lower with collapsed factor levels. This is not surprising, given that there is no relationship between foo$iv and the dependent variable, so reducing the degrees of freedom should always reduce AIC.

Of course, you should not simply test all possible combinations of factor levels for collapsing; that would be data dredging and even using AIC would invalidate subsequent inferences if you do not account for this process. Instead, let theory on how the data were generated guide you. And report what you did so consumers of your analysis can get a feeling how much iterative model fitting was involved.

added clarification
Source Link
Stephan Kolassa
  • 130.8k
  • 22
  • 265
  • 499

You shouldn't be rewriting your categorical data into dummies yourself. Just feed your categorical data into glm as-is; R will internally turn it into dummies. Then stepAIC() and similar methods can work correctly and either keep or remove the entire predictor, not only single levels.

Alternatively, you can look at whether collapsing different levels of your categorical variable improves matters, either using AIC or with Likelihood Ratio or Wald tests. As an example, let's create some toy data:

set.seed(1)
foo <- data.frame(dv=sample(x=c(0,1),size=100,replace=TRUE),
  iv=sample(LETTERS[1:5],size=100,replace=TRUE))
model <- glm(dv~iv,family="poisson",data=foo)
AIC(model)

Next, we collapse a couple of levels of foo$iv into single levels of a new predictor and fit a model with the new predictor:

foo$new.iv <- NA
foo$new.iv[foo$iv %in% c("A","B")] <- "a"
foo$new.iv[foo$iv %in% c("C","D")] <- "b"
foo$new.iv[foo$iv == "E"] <- "c"
AIC(update(model,.~new.iv))

AIC is lower with collapsed factor levels. This is not surprising, given that there is no relationship between foo$iv and the dependent variable, so reducing the degrees of freedom should always reduce AIC.

Of course, you should not simply test all possible combinations of factor levels for collapsing; that would be data dredging and even using AIC would invalidate subsequent inferences if you do not account for this process. Instead, let theory on how the data were generated guide you. And report what you did so consumers of your analysis can get a feeling how much iterative model fitting was involved.

You shouldn't be rewriting your categorical data into dummies yourself. Just feed your categorical data into glm as-is; R will internally turn it into dummies. Then stepAIC() and similar methods can work correctly and either keep or remove the entire predictor, not only single levels.

Alternatively, you can look at whether collapsing different levels of your categorical variable improves matters, either using AIC or with Likelihood Ratio or Wald tests.

You shouldn't be rewriting your categorical data into dummies yourself. Just feed your categorical data into glm as-is; R will internally turn it into dummies. Then stepAIC() and similar methods can work correctly and either keep or remove the entire predictor, not only single levels.

Alternatively, you can look at whether collapsing different levels of your categorical variable improves matters, either using AIC or with Likelihood Ratio or Wald tests. As an example, let's create some toy data:

set.seed(1)
foo <- data.frame(dv=sample(x=c(0,1),size=100,replace=TRUE),
  iv=sample(LETTERS[1:5],size=100,replace=TRUE))
model <- glm(dv~iv,family="poisson",data=foo)
AIC(model)

Next, we collapse a couple of levels of foo$iv into single levels of a new predictor and fit a model with the new predictor:

foo$new.iv <- NA
foo$new.iv[foo$iv %in% c("A","B")] <- "a"
foo$new.iv[foo$iv %in% c("C","D")] <- "b"
foo$new.iv[foo$iv == "E"] <- "c"
AIC(update(model,.~new.iv))

AIC is lower with collapsed factor levels. This is not surprising, given that there is no relationship between foo$iv and the dependent variable, so reducing the degrees of freedom should always reduce AIC.

Of course, you should not simply test all possible combinations of factor levels for collapsing; that would be data dredging and even using AIC would invalidate subsequent inferences if you do not account for this process. Instead, let theory on how the data were generated guide you. And report what you did so consumers of your analysis can get a feeling how much iterative model fitting was involved.

Source Link
Stephan Kolassa
  • 130.8k
  • 22
  • 265
  • 499

You shouldn't be rewriting your categorical data into dummies yourself. Just feed your categorical data into glm as-is; R will internally turn it into dummies. Then stepAIC() and similar methods can work correctly and either keep or remove the entire predictor, not only single levels.

Alternatively, you can look at whether collapsing different levels of your categorical variable improves matters, either using AIC or with Likelihood Ratio or Wald tests.