I am reading the an online book by Stef Van Buuren (link at bottom) regarding multiple imputation. In Section 3.2.1 he lists 4 different approaches to multiple imputation:

enter image description here

Later on in Section 3.3 he states: "The imputation methods discussed in Section 3.2 produce imputations drawn from a normal distribution."

I'm struggling to see how this is the case. Obviously I can see the error term added is drawn from a normal distribution. But surely the distribution of the imputations depend on the distribution of each of the predictor variables in X? Seeing as it is a linear regression, I would have thought the distribution of Y would be a combination of all of the distributions of the X's, as well as the error term.

SciKit learn's IterativeImptuter documentation also states: "The version implemented assumes Gaussian (output) variables. If your features are obviously non-normal, consider transforming them to look more normal to potentially improve performance"

I feel like this is because the default option for IterativeImputer draws the regression coefficients from a gaussian posterior distribution.

Based on all of the above I have the following questions:

  1. Where in the above equations is normality assumed, and why do the equations "produce imputations drawn from a normal distribution".
  2. Is the distribution of any imputed value determined by the model/equations used, which surely themselves depend on the distribution of X's, which are not necessarily normal?

Sci kit learnt docs: https://scikit-learn.org/stable/auto_examples/impute/plot_missing_values.html#sphx-glr-auto-examples-impute-plot-missing-values-py

Stef Van Buuren book: https://stefvanbuuren.name/fimd/sec-linearnormal.html


Under the usual assumptions for a linear regression model, the coefficient estimates have a multivariate normal distribution, as described in the coefficient covariance matrix. The predicted response, as a sum of normally distributed random variables, thus also has a normal distribution (Method 1). Method 2 just adds yet another (independent) normally-distributed term. Similar arguments apply to the other Methods. That is, multiple predictions for any particular missing data case will have a normal distribution.

That said, I think that the first highlighted used of "normal" in the quote might be in the sense of an "ordinary" linear model rather than in the technical sense of a "normal distribution." Be careful before you put too much weight on particular uses of words like "normal" that have both technical and colloquial meanings. Even the most careful author aided by a rigorous editor can still produce a written product with ambiguous usage of such words.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.