1
$\begingroup$

I have a short question. I am implementing Scikit-Learn in Typescript and currently blocked at understanding & implementing imputer (mean and regression strategies).

Based on the example given on Scikit-Learn page, http://scikit-learn.org/stable/modules/preprocessing.html#imputation

>>> import numpy as np
>>> from sklearn.preprocessing import Imputer
>>> imp = Imputer(missing_values='NaN', strategy='mean', axis=0)
>>> imp.fit([[1, 2], [np.nan, 3], [7, 6]])
Imputer(axis=0, copy=True, missing_values='NaN', strategy='mean', verbose=0)
>>> X = [[np.nan, 2], [6, np.nan], [7, 6]]
>>> print(imp.transform(X))                           
[[ 4.          2.        ]
 [ 6.          3.666...]
 [ 7.          6.        ]]

How does imputation against np.nan in [[np.nan, 2], [6, np.nan], [7, 6]] work?

  • Could you please explain it to me how it works? I would appreciate any equations and background knowledge that I need to understand this.
$\endgroup$
7
  • $\begingroup$ If you are asking about Scikit then your question is off topic here. If you are asking something more general about imputation, then please edit your question to reflect that. $\endgroup$
    – Peter Flom
    May 3, 2018 at 14:26
  • $\begingroup$ Is it fine to use Scikit Learn code as a reference? $\endgroup$ May 4, 2018 at 10:10
  • $\begingroup$ -reference- -> example* $\endgroup$ May 4, 2018 at 10:28
  • $\begingroup$ You can use code as a reference, but if you are trying to debug code, this is the wrong forum. $\endgroup$
    – Peter Flom
    May 4, 2018 at 10:45
  • $\begingroup$ Question: Don't you think it still belongs to this place because it's not really asking about implementation detail but asking about how it works? $\endgroup$ May 7, 2018 at 5:18

1 Answer 1

3
$\begingroup$

The Imputer is just calculating the mean for each column when fit is called. So column 1 has mean (1+7)/2 = 4 and column 2 has mean (2+3+6)/3 = 3.666....

The transform function just fills in the NaN fields with the column's mean value.

$\endgroup$
2
  • $\begingroup$ How about regression strategy? Could you explain that to me as well? Thanks in advance again $\endgroup$ May 2, 2018 at 23:58
  • $\begingroup$ There is no regression strategy in sklearn, but you could do this yourself by training a model using the other columns as the dataset and the column to be imputed as the target, nearest neighbor is a simple and fast way to do this (use the value of your closest neighbor(s) for each missing field). $\endgroup$ May 3, 2018 at 0:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.