17
$\begingroup$

I want to impute missing values of a dataset for machine learning (knn imputation). Is it better to scale and center the data before the imputation or afterwards?

Since the scaling and centering might rely on min and max values, in the first case the subsequent imputation might add new max / min values and tamper the scaled/centered data.

However, the imputation process might also profit from a scaled and centered dataset.

What do you think is better, and why?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ It is better to impute data during the machine learning process using EM algorithm. This way you are avoiding forcing a topology that will be inferior to the task at hand. $\endgroup$
    – Cagdas Ozgenc
    Commented Feb 18, 2015 at 18:00

2 Answers 2

Reset to default
8
$\begingroup$

It really depends on the Imputation technique being used. For example if we Impute using distance based measure (eg. KNN), then it is recommended to first standardize the data and then Impute. That is because lower magnitude values converge faster.

One idea could be using preprocess function from caret package. When you use method = knnImpute, it first center and scale the data before imputation.

preProcValues <- preProcess(data, method = c("knnImpute","center","scale"))

Improve this answer
$\endgroup$
1
  • $\begingroup$ Also if the imputation uses regression it will work better if the variables have been previously standardized. $\endgroup$
    – skan
    Commented Sep 18 at 19:43
8
$\begingroup$

Presumably, if you really need to center & scale the data, that should be done after imputation, as the imputation could influence on the correct center and scale to use!

Generally, the imputation should be the very first step in any analysis you do.

EDIT answer to comment by @Inon:

You say that imputation should preserve center & scale, and also standardization. Why? If the missing values truly are missing at random, maybe it does not matter much, but generally missingness might depend on other observed variables, and then estimates of mean and scale could be skewed by this pattern in the missingness. Imputation (better multiple imputation) is a way to fight this skewing. But if you do imputation after scaling, you just preserve the bias introduced by the missingness mechanism. Imputation is meant to fight this, and doing imputation after scaling just defeats this.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I'm not sure I understand your reasoning. We don't want imputed values to affect the center and scale of the data set. So wouldn't it be easier to ignore the missing values before they are replaced with valid values? It seems to me that imputation should follow enumeration of categorical features, standardization (of mean and variance) and vectorization (where relevant), and precede censoring, binning, and other numerical transformations. What do you think? $\endgroup$
    – Inon
    Commented Nov 9, 2016 at 23:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.