4
$\begingroup$

I am trying to compare imputation methods for an 81 samples x 407 variables data set with ~17% missing values. Some of the variables will be correlated, some highly, that is the nature of the data. I have already filtered variables out of the data set according to variance in quality control samples and number of missing values within treatment groups. None of the variables are categorical, all are numeric or integers. My script is:

mice(data, m=5, method="pmm")

And the error is:

Error in solve.default(xtx + diag(pen)) : 
  system is computationally singular: reciprocal condition number = 1.48341e-18

Other imputation functions work just fine i.e. median, k nearest neighbors, random forest. Why does mice fail?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Presumably you are trying to invert an non-invertible matrix. Possibly one of your variables is a linear combination of others of your variables $\endgroup$
    – Henry
    Commented May 24, 2016 at 6:56
  • $\begingroup$ Thanks @Henry, it is very likely the problem lies with the collinearity but variables that are highly correlated are of interest to me. Would this mean the other imputation methods I tried would result in low quality data or is it mainly a problem with predictive mean matching? $\endgroup$
    – Emma
    Commented May 24, 2016 at 20:07
  • 2
    $\begingroup$ You might want to read jstatsoft.org/article/view/v045i03/v45i03.pdf especially pages 22 26 and 42 which mention collinearity $\endgroup$
    – Henry
    Commented May 24, 2016 at 20:19
  • $\begingroup$ kaggle.com/c/house-prices-advanced-regression-techniques/… Will give you the answer for the problem $\endgroup$
    – user2279481
    Commented Mar 27, 2018 at 14:25
  • $\begingroup$ See both answers at stats.stackexchange.com/questions/76488/… $\endgroup$
    – rolando2
    Commented Mar 27, 2018 at 20:32

0

Reset to default

Browse other questions tagged or ask your own question.