3 edited tags; edited title
| link

using Using regression for imputing missing data

2 added 2110 characters in body
source | link

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.

EDIT 1:

To give some technical/theoretical background, in section 3.2.1 of the book "Flexible Imputation of Missing Data":

For univariate $Y$ we write lowercase $y$ for $Y$ . Any predictors in the imputation model are collected in $X$. Symbol $X_{obs}$ indicates the subset of $n_1$ rows of $X$ for which $y$ is observed, and $X_{mis}$ is the complementing subset of n 0 rows of $X$ for which $y$ is missing. The vector containing the $n_1$ observed data in $y$ is denoted by $y_{obs}$ , and the vector of $n_0$ imputed values in $y$ is indicated by $\dot{y}$. This section reviews four different ways of creating imputations under the normal linear model. The four methods are:

  1. Predict. $\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}}$ , where $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$ are least squares estimates calculated from the observed data. Section 1.3.4 named this regression imputation. In mice this method is available as "norm.predict".

  2. Predict + noise. $\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}} + \dot{\epsilon}$, where $\dot{\epsilon}$ is randomly drawn from the normal distribution as $\dot{\epsilon} \sim N(0, \hat{\sigma}^2)$. Section 1.3.5 named this stochastic regression imputation. In mice this method is available as "norm.nob".

  3. Bayesian multiple imputation. $\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$, where $\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$ and $\dot\beta_{0}$ , $\dot\beta_{1}$ and $\dot\sigma$ are random draws from their posterior distribution, given the data. Section 3.1.3 named this “predict + noise + parameters uncertainty.” The method is available as "norm".

  4. Bootstrap multiple imputation. $\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$, and where $\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$ and $\dot\beta_{0}$ , $\dot\beta_{1}$ and $\dot\sigma$ are the least squares estimates calculated from a bootstrap sample taken from the observed data. This is an alternative way to implement “predict + noise + parameters uncertainty.” The method is available as "norm.boot".

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.

EDIT 1:

To give some technical/theoretical background, in section 3.2.1 of the book "Flexible Imputation of Missing Data":

For univariate $Y$ we write lowercase $y$ for $Y$ . Any predictors in the imputation model are collected in $X$. Symbol $X_{obs}$ indicates the subset of $n_1$ rows of $X$ for which $y$ is observed, and $X_{mis}$ is the complementing subset of n 0 rows of $X$ for which $y$ is missing. The vector containing the $n_1$ observed data in $y$ is denoted by $y_{obs}$ , and the vector of $n_0$ imputed values in $y$ is indicated by $\dot{y}$. This section reviews four different ways of creating imputations under the normal linear model. The four methods are:

  1. Predict. $\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}}$ , where $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$ are least squares estimates calculated from the observed data. Section 1.3.4 named this regression imputation. In mice this method is available as "norm.predict".

  2. Predict + noise. $\dot{y} = \hat{\beta_{0}} + X_{mis} \hat{\beta_{1}} + \dot{\epsilon}$, where $\dot{\epsilon}$ is randomly drawn from the normal distribution as $\dot{\epsilon} \sim N(0, \hat{\sigma}^2)$. Section 1.3.5 named this stochastic regression imputation. In mice this method is available as "norm.nob".

  3. Bayesian multiple imputation. $\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$, where $\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$ and $\dot\beta_{0}$ , $\dot\beta_{1}$ and $\dot\sigma$ are random draws from their posterior distribution, given the data. Section 3.1.3 named this “predict + noise + parameters uncertainty.” The method is available as "norm".

  4. Bootstrap multiple imputation. $\dot{y} = \dot{\beta_{0}} + X_{mis} \dot{\beta_{1}} + \dot{\epsilon}$, and where $\dot{\epsilon} \sim N(0, \dot{\sigma}^2)$ and $\dot\beta_{0}$ , $\dot\beta_{1}$ and $\dot\sigma$ are the least squares estimates calculated from a bootstrap sample taken from the observed data. This is an alternative way to implement “predict + noise + parameters uncertainty.” The method is available as "norm.boot".

1
source | link

using regression for missing data

I have been reading about regression models for missing data imputation and I'm quite confused regarding the following: if I can perfectly predict the value of feature f2 using feature f1, why would I use f2? If both were real, would this mean that they are highly correlated, even if in a non-linear fashion? As far as I know, this class of imputation methods tries to predict a feature using another set of features.