Skip to main content
Cross Validated

Return to Question

added 2124 characters in body
Source Link
vdiddy
vdiddy
  • 119
  • 7
  1. if I run this through a multicollinearity check and get VIFs for one categorical variable being higher than 10, does it make sense to remove it?

WhatFor example:

      Feature    |        Variable       |  VIF  |
 =================================================
      Gender     |         Male          |   2
 -------------------------------------------------
      Weight     |  Weight (continuous)  |   1
 -------------------------------------------------
   Income Bucket |        0 - 50k        |   3
   Income Bucket |       51k - 75k       |   4
   Income Bucket |       76k - 100k      |   11 <--
   Income Bucket |       101k - 150k     |   1
 -------------------------------------------------
     Continent   |      North America    |   2
     Continent   |      South America    |   3
     Continent   |         Europe        |   4
     Continent   |          Asia         |   1
     Continent   |         Africa        |   1
     Continent   |        Australia      |   2

Should I wantdrop the categorical variable for Income Bucket = 76k- 100k and then proceed to know isfurther model selection?

  1. Furthermore, does it make sense if one or more (but not all) have p-values are not significant?

For example:

      Feature    |        Variable       |  P-Value  |
 =====================================================
      Gender     |         Male          |   0.002
 -----------------------------------------------------
      Weight     |  Weight (continuous)  |   0.001
 -----------------------------------------------------
   Income Bucket |        0 - 50k        |    0.05
   Income Bucket |       51k - 75k       |    1.65 <--
   Income Bucket |       76k - 100k      |    0.03
   Income Bucket |       101k - 150k     |   0.002
 -----------------------------------------------------
     Continent   |      North America    |    0.05
     Continent   |      South America    |   0.001
     Continent   |         Europe        |    11.2 <--
     Continent   |          Asia         |    0.01
     Continent   |         Africa        |    0.09
     Continent   |        Australia      |    0.1

Here, does it make sense to drop Income Bucket = 51k if I run this through a multicollinearity check and get VIFs for one categorical variable being higher than 10, does it make sense to remove it? Furthermore, does it make sense if one or more (but not all) have p-values are not significant?- 75k and Continent = Europe based on the P-values?

  1. How does the "missing" categorical variable (the one removed due to the (n - 1) rule) become interpreted once you drop a few of the ones that are included?

What I want to know is, if I run this through a multicollinearity check and get VIFs for one categorical variable being higher than 10, does it make sense to remove it? Furthermore, does it make sense if one or more (but not all) have p-values are not significant?

  1. if I run this through a multicollinearity check and get VIFs for one categorical variable being higher than 10, does it make sense to remove it?

For example:

      Feature    |        Variable       |  VIF  |
 =================================================
      Gender     |         Male          |   2
 -------------------------------------------------
      Weight     |  Weight (continuous)  |   1
 -------------------------------------------------
   Income Bucket |        0 - 50k        |   3
   Income Bucket |       51k - 75k       |   4
   Income Bucket |       76k - 100k      |   11 <--
   Income Bucket |       101k - 150k     |   1
 -------------------------------------------------
     Continent   |      North America    |   2
     Continent   |      South America    |   3
     Continent   |         Europe        |   4
     Continent   |          Asia         |   1
     Continent   |         Africa        |   1
     Continent   |        Australia      |   2

Should I drop the categorical variable for Income Bucket = 76k- 100k and then proceed to further model selection?

  1. Furthermore, does it make sense if one or more (but not all) have p-values are not significant?

For example:

      Feature    |        Variable       |  P-Value  |
 =====================================================
      Gender     |         Male          |   0.002
 -----------------------------------------------------
      Weight     |  Weight (continuous)  |   0.001
 -----------------------------------------------------
   Income Bucket |        0 - 50k        |    0.05
   Income Bucket |       51k - 75k       |    1.65 <--
   Income Bucket |       76k - 100k      |    0.03
   Income Bucket |       101k - 150k     |   0.002
 -----------------------------------------------------
     Continent   |      North America    |    0.05
     Continent   |      South America    |   0.001
     Continent   |         Europe        |    11.2 <--
     Continent   |          Asia         |    0.01
     Continent   |         Africa        |    0.09
     Continent   |        Australia      |    0.1

Here, does it make sense to drop Income Bucket = 51k - 75k and Continent = Europe based on the P-values?

  1. How does the "missing" categorical variable (the one removed due to the (n - 1) rule) become interpreted once you drop a few of the ones that are included?
Source Link
vdiddy
vdiddy
  • 119
  • 7

Feature Selection with Categorical Variables: Multicollinearity and Statistical Significance

Building a logistic regression model with three categorical features and one continuous. For simplicity, let's say I have the following features and variables:

      Feature    |        Variable
 ========================================
      Gender     |         Male
 ----------------------------------------
      Weight     |  Weight (continuous)
 ----------------------------------------
   Income Bucket |        0 - 50k 
   Income Bucket |       51k - 75k 
   Income Bucket |       76k - 100k 
   Income Bucket |       101k - 150k 
 ----------------------------------------
     Continent   |      North America 
     Continent   |      South America
     Continent   |         Europe
     Continent   |          Asia
     Continent   |         Africa
     Continent   |        Australia

Recall that I am using (n - 1) for the number of variables so there is no variable for Gender = Female, Income Bucket = 100k+, or Continent = Antarctica.

What I want to know is, if I run this through a multicollinearity check and get VIFs for one categorical variable being higher than 10, does it make sense to remove it? Furthermore, does it make sense if one or more (but not all) have p-values are not significant?

Remember that this is for a logistic regression but I am okay with knowing how it would work for linear as well. And would another method be more suited for this?

Thank you!