I have what I'll call L-shaped panel data:

1  2  3  4 NA NA NA NA NA  
1  2  3  4 NA NA NA NA NA  
NA 2  3  4 NA NA NA NA NA  
1  2  3  4 NA NA NA NA NA  
1  2  3  4 NA NA NA NA NA  
1  2  3  4  5  6  7  8  9  
1  NA 3  4  5  6  7  8  9  
1  2  3  4  5  6  NA 8  9  

The values in the top right block are structurally missing and will not be used in my analyses. The scattered NA's in the rest of the data frame need to be imputed. I'm not sure what the best way to do this is.

Context: I am analyzing a survey in which 30 of the questions appeared in all survey 16 waves and another 20 questions were added for just the last two waves. These last 20 questions would have been meaningless in earlier waves because the events they refer to hadn't happened yet. Therefore, I don't need to impute those values, but I do need to impute the missing-at-random values scattered throughout the rest of the dataset. How should I go about doing this?

Idea 1: Create two subsets: one with the almost complete initial columns, and one with the almost complete final rows. Use multiple imputation on each subset separately and combine the results. Downside: Discards information, particularly in the second imputation. Not sure if combining them this way violates the assumptions underlying Rubin's rules.

Idea 2: Use multiple imputation on the entire dataset. Downside: Whereas in the first approach about 25% of the values are missing in each subset, here about 50% of the values are missing. This slows things down a lot computationally given that I have 50 columns and 13,000 rows. Not sure if there are theoretical problems as well.

What do you recommend? (I'm open to using Amelia, mi, mice, mlmi or any other package etc.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.