2
$\begingroup$

I have a model with the following parameters:

  • Groups: factor - 4 levels (base level = control group)
  • Time: numerical
  • Label = factor - 3 levels (base level = control group)
  • price = numerical (5 different values; from a Likert scale)

The problem is that the base level of variable 'Groups' is perfectly collinear with the variable Time because, in the control condition, no values for Time were collected. That means that for Groups = 'control group', Time is always '0'.

This introduces singularities or perfect collinearity in my regression model, meaning I cannot interpret it correctly.

Do you have any suggestions for helping me out? Recollecting data is not an option, unfortunately. It was too costly and there is too many time constraints.

Improve this question
$\endgroup$
5
  • $\begingroup$ Why do you encode missing values as 0? It should be NA, i.e., a special value that indicates unknown values and propagates correctly in three-valued logic. If you do not want to omit the Time variable, you can try "imputing" the missing values, e.g. wth a technique called "multiple imputation". $\endgroup$
    – cdalitz
    Commented Jun 7, 2022 at 4:39
  • $\begingroup$ Hi cdalitz! Thanks for your reply. Since there is a 1:1 relationship between "missing values for variable Time" and "variable Groups", wouldn't multiple imputation be biased? Including NAs removes the variable from my regression model (in R) altogether unfortunately [I'm using: summary(lm(x ~ y, data = data))]. $\endgroup$
    – Hans.nl
    Commented Jun 7, 2022 at 21:13
  • $\begingroup$ Obviously, you cannot include the predictor Group in the imputation process. Out of curiosity, I have just tried a regression on the Iris dataset with Sepal.Length set to zero for Species == "setosa", and the results look reasonable with an $R^2$ of over 95%, which is only marginally smaller than with the original values. In other words, the preporcessing step of transforming *Sepal.Length still produces an excellent prediction model. What exactly is the problem in your case? $\endgroup$
    – cdalitz
    Commented Jun 8, 2022 at 5:43
  • $\begingroup$ Hmm, that doesn't sound to bad. So the variable GROUPS has 4 levels with 0 being the reference and control group. When I run a regression on this factor variable, I end up with only 2 instead of 3 coefficients, meaning that the regression ingnores my 0 level, and instead uses Group = 1 as my reference level. This doesn't change after revelling. $\endgroup$
    – Hans.nl
    Commented Jun 8, 2022 at 9:06
  • $\begingroup$ The only thing I can think of right now is to insert a random number between 0 and 1 seconds, but I have no idea what that would mean in terms of statistical interpretation. How exactly did you run your model that still had a high R? $\endgroup$
    – Hans.nl
    Commented Jun 8, 2022 at 9:21

1 Answer 1

Reset to default
0
$\begingroup$

First notice that setting the unknown value to zero is an arbitrary choice, and you will obtain wildly different values if you make a different choice (e.g. $42$ or $6\times 10^{23}$). It is more reasonable to fill the unknown values with an actually possible and even likely value, e.g., the mean of all other values. "Mean imputation" is a fast and simple method, but it obviously ignores dependencies among the predictors. In your case it introduces the additional assumption that the mean Time in the Control group is identical to the other groups. OTOH, as a working assumption it might be reasonable, because if even under this assumption you can find significant differences, there might be some effect.

As you have not provided test data, let's try it with the iris dataset in R:

> x <- iris
> x$Sepal.Length[x$Species == "setosa"] <- NA
> x$Sepal.Length[x$Species == "setosa"] <- mean(x$Sepal.Length, na.rm=TRUE)
> fit1 <- lm(Petal.Length ~ ., x)
> summary(fit1)
Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)       -2.75128    0.32251  -8.531 1.83e-14 ***
Sepal.Length       0.64868    0.04754  13.646  < 2e-16 ***
Sepal.Width        0.01045    0.06993   0.149    0.881    
Petal.Width        0.46911    0.11557   4.059 8.05e-05 ***
Speciesversicolor  2.50971    0.15925  15.760  < 2e-16 ***
Speciesvirginica   3.04826    0.22329  13.651  < 2e-16 ***
---
Multiple R-squared:  0.9812,    Adjusted R-squared:  0.9805

Note that there is no problem with collinearity, because Sepal.Length is different for some other equal values of Species. This becomes a problem, however, if we assume different slopes for different species:

> fit2 <- lm(Petal.Length ~ . + Sepal.Length:Species, x)
> summary(fit2)
Coefficients: (1 not defined because of singularities)
                               Estimate Std. Error t value Pr(>|t|)    
(Intercept)                    -3.01217    0.38714  -7.781 1.31e-12 ***
Sepal.Length                    0.68869    0.05778  11.919  < 2e-16 ***
Sepal.Width                     0.01258    0.06983   0.180    0.857    
Petal.Width                     0.48158    0.11584   4.157 5.52e-05 ***
Speciesversicolor               3.14623    0.54802   5.741 5.43e-08 ***
Speciesvirginica                3.01398    0.22471  13.413  < 2e-16 ***
Sepal.Length:Speciesversicolor -0.10707    0.08822  -1.214    0.227    
Sepal.Length:Speciesvirginica        NA         NA      NA       NA
---
Multiple R-squared:  0.9814,    Adjusted R-squared:  0.9806

Fortunately, the R implementation of lm automatically detects the perfect collinearity between predictors and removes the problematic one.

For a more detailed overview on imputing missing values, see the online book "Flexible Imputation of Missing Values" by Stef van Buuren (2018).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.