How to analyze a dichotomous outcome with 50% missing data?

Question

I am researching predictors of dropout from a training program. I want so to see if personality traits add incremental variance above well-established predictors like age, fitness, and education. So, in the end, I want to do a hierarchical logistic regression and compare the difference in R². The (simplified) full model to predict dropout will look something like this:

dropout = age + education type + fitness level + personality trait A … + personality trait J

I do not expect any interactions. Dropout is a dichotomous variable (yes/no). Education type is a categorical variable, fitness level ordinal, the personality traits are continuous and normally distributed.

I measured potential predictors at T1 (age, fitness, education, personality traits) and the outcome “dropout” a few months later at T2. While there is less than 1% missing data for the predictors in T1, about 50% of the outcome in T2 is missing. In other words, the questionnaire in T1 was answered by 1000 persons, and my follow-up call in T2 (whether people still do the training) by 500. Out of these 500, 460 people kept up the training and 40 people dropped out. I am not sure how I should handle the focal analysis of my question (are personality traits useful predictors) and this amount of missing data. I am usually working with R and consider two options:

Option 1: I consider using multiple imputation to fill in the missing data (R package MICE). The data suggests a MAR mechanism, where the missingness depends on the personality of the participants (undisciplined people fill out T2 significantly less). I could use this information to tailor the imputations and do a sensitivity analysis to estimate what effect an alternative NMAR mechanism would have. Then I could use a logistic regression model to answer my main question, namely which variables from T1 best predict dropout.

Option 2: I think I also could build a structural equation model to answer my question and use FIML to handle missing data (R package lavaan). This option is attractive because I will build SEMs anyway to analyze personality variables for a different research question. SEM also considers measurement errors; I think those would go unnoticed in option 1. However, I am not sure if SEM and ML can handle this non-normal outcome with 50% missing data. Also, I am used to interpreting the effect of predictors in terms of R² and ORs, would a similar quantification be available in a structural equation model?

Which option would you recommend to me and why?

To clarify: At T2 I just measured the outcome (dropout yes/no), none of the other variables. The amount of missing data is high because, contrary to T1, the participants were contacted via phone or email at home. In T1, everyone was gathered in a classroom setting and probably felt more committed. This was not possible to do at T2.

Answer 1

Your description implies that you committed the mortal sin of not pre-specifying the final model in the sense that you tried different models in a way not unlike stepwise variable selection does (mislabeling this as hierarchical modeling). This will ruin all aspects of inference, especially resulting in a falsely small $\hat{\sigma}$. More to your question, this is not best called a missing data problem but is a problem with non-random non-response. Surveys that examine changes over time (such as pre-post analysis) must ensure that all people included at T1 also respond at T2 otherwise results can have a major bias.

You might do a conditional analysis of persons who had data in both periods. But this is very conditional.

If you treat this as a missing data problem, multiple imputation will not add much efficiency over doing a complete case analysis when the main variable missing is the dependent variable.

Think hard about whether this study design is adequate to answer your main questions of interest. It may not be.

Answer 2

If you want to predict what happens at T2 from data at T1, you could run a three classes model with "dropout", "no dropout", and "T2 missing" as the classes.

Note that data with missing outcome will not help you to predict the outcome from the predictors. Imputing the outcome is pointless, as this doesn't increase the amount of information of how the predictors work on the outcome. "The data suggests a MAR mechanism" - there is no way to distinguish MAR from MNAR based on the data.

Answer 3

First, I don't see how variables at time 2 can be sensibly used to predict dropout at time 2. So, I don't think you need to even worry about the missing data. I'm not sure why you collected it. You seem to know who dropped out, even if they have missing data at time 2, since you refer to that information.

Second, I agree with Frank Harrell about the perils of doing variable selection the way you seem to be planning to do it. At a minimum, if you are planning this, then do a train and test set -- that is, divide the data in two and build your model on the training set, then report the results from running the same model on the test set.

Third, right now, from your description, I see no need for SEM. You seem to have a relatively straightforward logistic regression problem. But maybe there are things you haven't said that make SEM worthwhile. In general, I am not a big fan of SEM. Of course it can be useful when done well, but it's very easy to go wrong, and very easy to cheat by adjusting things endlessly until they work. Do you have some sort of latent variable?

Fourth, be sure to check for collinearity among the 10 personality traits. These traits are usually related to each other and can cause problems.

Finally, I don't know why you are ruling out interactions.