1
$\begingroup$

I'm running a regression in RStudio with a continuous outcome and several predictors. One of my predictors is categorical and has 5 categories. Its classed as factor. When I run my regression all the categories but two disappear. I've checked my contrasts and know that the 'Home' category is being used as a reference category, but can't for the life of me figure out why my regression isn't showing the rest of the categories (multiple and other).

I'm aware I could code each category as a dummy variable, but I'd prefer not to since I'm trying to show students how to do this and am not sure why this is happening. I'm probably doing something stupid, but if anyone has ideas, that would be greatly appreciated!

USwave1$jobplace <- recode(USwave1$jbpl, "1='Home';2='Office';3='Travels';4='Multiple';5=NA;97='Other'")
table(USwave1$jobplace)

Home  Multiple   Office    Other  Travels 
547     1734     19973      110     2146 
 

contrasts(USwave1$jobplace)
 
         Multiple Office Other Travels
Home            0      0     0       0
Multiple        1      0     0       0
Office          0      1     0       0
Other           0      0     1       0
Travels         0      0     0       1

model1 <- lm(data = USwave1, formula = "jbsat ~ jobplace + a_age_dv + male + commute + paytype + hours")
summary(model1)

Call:
lm(formula = "jbsat ~ jobplace + a_age_dv + male + commute + paytype + hours", 
    data = USwave1)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.7483 -0.4544  0.5777  0.8533  2.7738 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)      5.2635590  0.0635886  82.775  < 2e-16 ***
jobplaceOffice  -0.1669841  0.0431742  -3.868  0.00011 ***
jobplaceTravels -0.1463690  0.0564316  -2.594  0.00950 ** 
a_age_dv         0.0076257  0.0008414   9.063  < 2e-16 ***
male            -0.1201247  0.0227431  -5.282 1.29e-07 ***
commute         -0.0025803  0.0004612  -5.595 2.23e-08 ***
paytypeSalaried  0.2036349  0.0234411   8.687  < 2e-16 ***
hours           -0.0044555  0.0010723  -4.155 3.27e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.508 on 20815 degrees of freedom
  (30171 observations deleted due to missingness)
Multiple R-squared:  0.01241,   Adjusted R-squared:  0.01208 
F-statistic: 37.37 on 7 and 20815 DF,  p-value: < 2.2e-16 ```

$\endgroup$
2
3
$\begingroup$

There is a clue that something may be off, which is embedded in the model summary:

(30171 observations deleted due to missingness)

That seems to be quite a large number of observations to be deleted from the model, so you need to investigate what is going on.

First, I would start with a simple summary(USwave1) to see the extent of data missingness and in particular which predictor variables suffer from it. Also, does the response variable also include missing values?

Next, I would then get a handle on how many rows of the dataset including the relevant variables for your modelling include no missing data values and how many include at least one missing data value:

library(dplyr)

USwave1sub <- select(USwave1, jbsat, jobplace, a_age_dv, 
                     male, commute, paytype, hours)
                     

nrow(na.omit(USwave1sub)

nrow(USwave1sub)- nrow(na.omit(USwave1sub))

When you fit an lm() model to a dataset where the response and/or predictor variables include missing values, R will drop all the rows of that dataset which include at least one missing values. For this reason, I would also investigate how many categories of jobplace are still represented in your data after the rows with missing values were excluded. Try something like this:

USwave1sub_nonmiss <- na.omit(USwave1sub)

USwave1sub_nonmiss$jobplace <- droplevels(USwave1sub_nonmiss$jobplace)

table(USwave1sub_nonmiss$jobplace, exclude = NULL) 

Compare the results of the last command against what you would get before dropping any missing values from your dataset:

table(USwave1sub$jobplace, exclude = NULL)

If you see certain categories of jobplace disappear after missing values in the dataset were dropped, then you know the source of your predicament: those categories happened to correspond to missing values in other variables included in your model, so they were dropped by R as it grappled with the data missingness issue.

For more on missing data exploration and visualization, you can check out the lovely naniar package by Nicholas Tierney: https://cran.r-project.org/web/packages/naniar/vignettes/getting-started-w-naniar.html.

$\endgroup$
1
  • 1
    $\begingroup$ Ah, mystery solved! Your intuition was correct, the numbers in the last two bits of your code matched perfectly, so R is dropping missing data from other variables included in my model and its essentially erasing those two categories. Thanks for showing me that neat code as well! I would have never caught that on my own. Thank you so much for your help! $\endgroup$ – user302632 Nov 30 '20 at 13:20
1
$\begingroup$

The code posted above by Isabella shows why my categories disappeared.

The categories which were dropped corresponded to missing values in other variables included my model.

$\endgroup$
1
  • $\begingroup$ If @Isabella helped you then it is best to accept her answer using the tick symbol (US=check mark). $\endgroup$ – mdewey Nov 30 '20 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.