2
$\begingroup$

I have a dataset with 10 independent variables and one response variable. They are all in integer types. V1 - V10 are numeric values rounded to closest integer values which are always in range of [1, 11], Response is a categorical data represented by integers 0, 1, 2, 3, like such:

'data.frame':   2997 obs. of  11 variables:
 $ V1      : int  2 5 4 2 10 4 2 11 8 7 ...
 $ V2      : int  4 5 7 1 5 9 1 3 2 2 ...
 $ V3      : int  2 4 5 5 4 4 2 7 4 10 ...
 $ V4      : int  6 2 4 7 8 2 10 9 2 7 ...
 $ V5      : int  5 8 2 11 4 6 11 9 2 10 ...
 $ V6      : int  7 5 4 3 7 6 2 5 2 1 ...
 $ V7      : int  1 3 10 7 3 8 2 2 4 3 ...
 $ V8      : int  2 1 3 1 6 8 5 3 8 6 ...
 $ V9      : int  6 7 9 10 7 4 1 2 6 2 ...
 $ V10     : int  8 1 5 7 3 3 3 6 10 1 ...
 $ Response: int  0 0 1 1 1 1 0 1 1 1 ...

They have little linear relationship with each other. In other words, the pairwise correction are nearly 0. Like such:

         V1   V2   V3   V4   V5   V6   V7   V8   V9  V10 Response
V1        1 0.03 0.05 0.01 0.03 0.05 0.05 0.05 0.04 0.06     0.28
V2             1 0.05 0.03 0.02 0.04 0.01 0.03 0.09 0.07     0.35
V3                  1 0.02 0.05 0.03 0.01 0.02 0.03 0.06     0.26
V4                       1 0.04 0.02 0.05 0.02 0.01 0.08     0.27
V5                            1 0.06 0.07 0.01 0.06 0.07      0.3
V6                                 1 0.02 0.04 0.07 0.06     0.33
V7                                      1 0.08 0.04 0.07     0.34
V8                                           1 0.04 0.03     0.33
V9                                                1 0.01     0.35
V10                                                    1     0.37
Response                                                        1

An example of a scatter plot for a pair variables would look like such: Scatter plot for V1 and V2

If I added some noise in them, it would look like such: Scatter plot for V1 and V2 with added noise

Among those variables, say, V1, V3, and V4, have missing values. My task is to build a regression model to impute those missing values.

My thought on tackling this task is:

  1. Remove all rows containing missing values. (Not many, so safe to delete)
  2. Make one of the variable containing missing values as the target variable and the rest as predictors. For example, V1 as target vaiable, V2 - Response to be predictors.
  3. Fit a regression model to impute the missing value

So I first tried linear regression with lm() function, turns out the model has really poor adjusted R-squared value, as such: Call: lm(formula = V1 ~ ., data = dt.cv)

Residuals: Min 1Q Median 3Q Max -7.2258 -1.9623 -0.0097 1.9256 6.1101

Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 9.54617 0.33450 28.538 < 2e-16 * V2 -0.15494 0.01925 -8.051 1.18e-15 V3 -0.08019 0.01818 -4.411 1.06e-05 V4 -0.12805 0.01852 -6.913 5.77e-12 V5 -0.12606 0.01872 -6.733 1.98e-11 V6 -0.12747 0.01935 -6.587 5.28e-11 V7 -0.12858 0.01915 -6.715 2.25e-11 V8 -0.12629 0.01926 -6.557 6.44e-11 V9 -0.14905 0.01968 -7.574 4.79e-14 V10 -0.13063 0.01964 -6.649 3.49e-11 *

Response 1.97108 0.09525 20.694 < 2e-16 ***

Signif. codes: 0 '' 0.001 '' 0.01 '' 0.05 '.' 0.1 ' ' 1

Residual standard error: 2.575 on 2986 degrees of freedom Multiple R-squared: 0.1374, Adjusted R-squared: 0.1345 F-statistic: 47.57 on 10 and 2986 DF, p-value: < 2.2e-16

Then I turned all variables into factors, hoping to fit a multinomial logistic regression model with glmnet() function. However the plot of the cv.glmnet(..., type.measure='class') yields very high misclassification rate,

So both linear regression and multinomial logistic regression(classification) won't be able to give a reasonable good imputation for the missing value.

How am I going to impute missing value with regression model for these low-correlated data? .

$\endgroup$
  • 2
    $\begingroup$ You may want to consider why you would impute the missing values in this dataset. As you said, you have very little information to predict the missing data from and there are only a few rows that actually contain missing data. The benifit of imputation seems to be quite minimal.You can look into the missing data mechanism, i.e. can you predict when a variable is missing from the values of the other variables? If you can, you might be able to say something about the possible effect of deleting these cases, but if there are only a few missing observations then the effect might not be that large. $\endgroup$ – Niek Oct 6 '16 at 16:57
  • $\begingroup$ If these are also time series, you could use univariate time series imputation methods. These work with inter-time correlations of one variable instead of inter-variable correlations. Otherwise, if the variables are not correlated at all....using column means/median,...might make sense. Also look at the comment of Niek. $\endgroup$ – stats0007 Nov 16 '16 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.