# Feature selection on large file with missing categorical and numerical data in R

I know that there have been similar questions but most of them have not worked, this is why I start this question.

I have a very large dataset (around 2,500,000 records) with approximately 100 variables, most of which are categorical and some of them numerical. There are a lot of missing values which are probably MCAR/MAR (the data come from an automated log file). Also, some categorical variables have a lot of different values (e.g. city attribute). I am trying to perform feature selection to find the most informative variables with regard to a discrete numerical target variable.

I have two problems:

i) How to fill in the missing values: most of the standard packages' functions get stuck (i.e. mice, hmisc). Should I use just the mode/median or fill in some default "unknown" value? If I decide to take a subset of the rows to decrease the size, how small should this subset be - if there are any empirical rules?

ii) Most of the feature selection techniques I have tried (fscaret, fselector, boruta) after using some simple imputation technique either hang or give me an error. I believe this has to do with the fact that some categorical variables have many possible values. What are some practical ways to deal with this?

• What is the RAM available prior to loading the data into R and RAM available after loading? – Sergey Bushmanov Oct 12 '15 at 18:22
• Thank you, I will update once I have the number. It takes a while to load the data. – user90772 Oct 13 '15 at 17:32
• If you suspect getting an error is a matter of imputation algo, you may try imputing on a fraction of data, say 1/10, but my suspicion you do not have enough RAM cause I estimate your data size is in Gb's – Sergey Bushmanov Oct 13 '15 at 17:57
• Thank you again, I have the same suspicion, but since I am a complete beginner, I guess I need to use some stratified sampling technique to get a representative sample, right? – user90772 Oct 13 '15 at 18:06
• Keep it simple. Use stratified sampling, if you can, to make your sample representative for imputation techniques you apply. My point, again, your data set appears a little too big, at least for my laptop. On top of that, R is not very efficient in memory management. Make it sure your imputation algo works on a smaller sample, and then try increasing the size. – Sergey Bushmanov Oct 13 '15 at 18:35

I had a similar problem. I used the $impute$ (from Hastie and Tibshirani) package that does K-Nearest Neighbor imputation. It was designed for micro-arrays, that means gene expression data which is continuous. But if you convert your predictors, all of them to numeric, it does a nice job and very fast.
source("bioconductor.org/biocLite.R") biocLite("impute")
• Feature selection: the "LASSO" will be very suitable for this job. I am a big fan of Hastie and Tisbshirani. Use the $glmnet$ package. It will require patience. – Chamberlain Foncha Oct 13 '15 at 9:00