Coefficients not defined because of singularity - NOT because of dummy code error or multicollinearity

Question

Variables are being removed from my analysis. I've looked at other questions for similar problems, but the answer always seems to be an error when creating dummy codes (i.e., not creating n-1 dummy codes for categorical variables) or a suggestion of multicollinearity.

For this analysis, 3 variables are dichotomous and the rest are continuous, so my problem shouldn't be caused by an error in creating dummy codes. Furthermore, when I run a basic correlation analysis, none of my correlations are super high. I would be most concerned by the correlations of .84 and .77 for x5. However these are correlations with variables which have also been dropped (coefficients NA in output).

I really appreciate any help that anyone can give!

library(psych)

data %>%
   corr.test()

       x1     x2     x3     x4     x5     x6     x7     x8     x9     x10    x11    x12
x1     1.00

x2     0.02   1.00

x3     0.02   0.63   1.00

x4     0.18   0.65   0.60   1.00

x5     0.09  -0.01   0.03   0.04   1.00

x6     0.10  -0.06  -0.15  -0.06   0.84   1.00

x7     0.05   0.07   0.02   0.03   0.77   0.73   1.00

x8     0.05   0.00   0.04   0.16   0.69   0.68   0.69   1.00

x9    -0.06   0.03   0.05  -0.04  -0.25  -0.25  -0.21  -0.16   1.00

x10   -0.20   0.01  -0.12  -0.30  -0.24  -0.19  -0.06  -0.02   0.14   1.00

x11    0.12   0.40   0.49   0.50   0.19   0.13   0.21   0.16  -0.08  -0.20   1.00

x12    0.00  -0.03  -0.07  -0.05   0.01   0.04   0.13   0.21  -0.04   0.06  -0.05   1.00


lm(formula = x12 ~ ., data = data)

(Intercept)  x1       x2        x3       x4  x5  x6  x7  x8  x9  x10      x11

    4.12583  0.19116  1.78860  -5.27225  NA  NA  NA  NA  NA  NA  0.05925  NA

linmod <- lm(formula = x12 ~ ., data = data)
summary(linmod)

Residuals:
      Min        1Q    Median        3Q       Max 

-0.250136 -0.016327  0.000272  0.007364  0.249864 

Coefficients: (7 not defined because of singularities)

            Estimate   Std. Error  t value  Pr(>|t|)    

(Intercept) 4.12583    1.09878     3.755    0.007122 ** 

x1          0.19116    0.11351     1.684    0.136037    

x2          1.78860    0.23965     7.463    0.000142 ***

x3         -5.27225    0.79155    -6.661    0.000288 ***

x4          NA         NA          NA       NA    

x5          NA         NA          NA       NA    

x6          NA         NA          NA       NA    

x7          NA         NA          NA       NA    

x8          NA         NA          NA       NA    

x9          NA         NA          NA       NA    

x10         0.05925    0.04822     1.229    0.258877    

x11         NA         NA          NA       NA    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1359 on 7 degrees of freedom
  (110839 observations deleted due to missingness)
Multiple R-squared:  0.9195,    Adjusted R-squared:  0.8736 
F-statistic:    20 on 4 and 7 DF,  p-value: 0.0006233

Answer 1

I suspect this indicates the problem

  (110839 observations deleted due to missingness)

You've got 7 residual degrees of freedom, and five parameters, which means you must have only 12 observations without missing data to be used in the regression. With only 12 observations, it's not surprising to get singularities.

The 'singularity' means, for example, that some linear combination of the intercept, x1, x2, x3 and x4 is perfectly collinear with x5. Another linear combination is perfectly collinear with x6, and so on.

When the number of observations is much larger than the number of predictors, this is usually due to some obvious structure in the data, such as dummy variable coding, or proportions that have to add to 1. With a small number of observations, especially if some are discrete, it can easily happen by chance, as it seems to have done here.

Answer 2

Identifiability appears to be your problem, and not necessarily issues concerning correlatedness of your input variables. You have too few degrees of freedom and little hope of uniquely estimating all of the parameters in your model.