1
$\begingroup$

Here is the situation: I have an individual level data set $X$ where each row is a person $i$ and each column denote characteristics of $i$. The problem is that my data is missing an important variable, lets call this $z_{i}$.

To resolve this situation I am considering using data imputation. Note, I have not done this before so I am: (a) a bit unsure of how it actually works in practice, (b) when it can be considered a valid thing to do (and when it is not appropriate). Information pertaining to each of these, or both would therefore be valuable.

My perception of the approach, and proposed way of proceeding is the following (please let me know if this is wrong): I do not have data on $z$ in my dataset, but I have access to other data from which I could devise a empirical CDF $F(z)$. Next I would simply draw from this distribution and crudely impute these data into a new column in my dataset $X$. Is this the general approach taken?

I understand that a more appropriate approach would be to devise the conditional empirical cdf $G(z|X’)$ and to sample form this. I can actually do that because I have a separate sample where a proxy for $z$ is available--call it $z'$, so I could probably use this to get $G(z'|X)$. Would I next simply draw from these conditional distributions in order to construct the new imputed variable in the old dataset $X$?

I understand that these questions are rather naive, and I understand that any procedure of this kind must come with its own set of assumptions that will restrict when it is and is not appropriate, but the suggestions above are simply my take on the "intuition" behind the approach, and I firstly want to make sure that this is correct. Second, any information pertaining to the assumptions required on the data would be much appreciated too.

$\endgroup$
3
  • $\begingroup$ Are values of z missing for all individuals, or only for some? Also, do your "other data" contain values of the variables in your main data set? $\endgroup$
    – EdM
    Sep 2, 2014 at 16:53
  • $\begingroup$ @EdM, z is missing for all individuals, and Yes--the "other data" contains the all the values of the variables in the main data. $\endgroup$
    – Seb
    Sep 2, 2014 at 18:23
  • 1
    $\begingroup$ This depends on what you want to do with the imputed data set, which the question doesn't specify. $\endgroup$ Nov 17, 2023 at 14:14

1 Answer 1

2
$\begingroup$

If your other data and your new data set X are both random samples from the same underlying population and both have data on the same variables (except for variable z in set X), then you could combine all the data and perform multiple imputation to estimate the missing data in set X. You might also be able to include your separate sample with values for the proxy z' if that data set also has values on the other variables.

As you recognize in your description of the "more appropriate approach," treating z as independent of the other variables is not appropriate; you need to consider its multivariate relations to the other variables. Multiple estimates based on the multivariate empirical distribution and pooling of analyses among the imputations is crucial. Stef van Buuren's Flexible Imputation of Missing Data is a superb source on both theoretical background and practical use.

If, however, your new data set X is not really comparable to your other data sets, then you really shouldn't combine the data sets in this way and I don't know of a reliable way to impute the missing data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.