I have a large dataset comprising survey responses throughout a time period of three to four decades (or in other words, slightly above 20 waves). Keep in mind that these are longitudinal data and not panel data. This means, one row equals one respondent in reality. The dataset, as a whole, features roughly around 50% of missing cases: that is owed to (a) slightly different sets of questions per survey wave, and (b) the usual non-response.

To avoid shrinking down the overall dataset too much and to ensure that I can keep as many survey waves in the analysis as possible, I would like to use MI and especially the {caret} package in R which comes with versatile MI methods, such as preProcess(data, "knnImpute"); predict(...).

1.) From a theoretical and methodological perspective, is there anything I need to keep in mind when applying MI on a longitudinal dataset that comprises different sets of survey questions, some of which simply haven't been asked and therefore contain a lot of missing cases? Is it at all possible to impute values for questions that have been asked in 10 waves (missing: 0%) but not in wave 11 (missing: 100%)? Or do I need to avoid imputing values for questions that have not been asked entirely?

2.) When imputing values for questions that have been asked, should I focus always on the same wave or can I impute values in case of "non-response" induced missingness based on a larger model that draws on the whole dataset? If so, how would I implement this, using {caret}? The reason I am asking this is that I'm cognizant of the data structure (longitudinal survey data structured into 20+ waves), and hence want to be sure to not commit any methodological crimes, yet I also lack the experience in applying MI particularly when it comes to data like these.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.