3
$\begingroup$

Is it appropriate to include intermediate outcomes in a predictive model?

It is quite clear that one should not control for post-treatment variables / intermediate outcomes when the goal is causal inference, but I wasn't sure if the same advice should hold when one's goal is to build a model for prediction.

Here is some context for my question: I'm trying to build a model that predicts if a college student will earn a bachelor's degree within 6 years of high school graduation using a large observational data set. I have data on students' high school variables (HS GPA, test scores, number activities participated in, etc.), some data on the students' college experiences (delayed enrollment in college, full-time / part-time status, transferred within two years of enrolling), as well as data on the characteristics of the college they attend. In other words, I have student level and institutional level data. I would account for the nesting of students within a particular institution.

Some have told me that the student level data on college experiences are intermediate outcomes and I shouldn't include them in the model. It isn't clear to me if I should / could include the college experience variables (which could be considered intermediate outcomes) in the predictive model, and if I do include them, how they should be treated.

Improve this question
$\endgroup$
2
  • $\begingroup$ Can you say what you are going to use the model for? I am skeptical that you are really only interested in prediction. $\endgroup$
    – Bill
    Commented Feb 14, 2014 at 18:55
  • $\begingroup$ Here's my idea. I have comparable and consistent data from three cohorts of students spanning the 1980s, 1990s, and 2000s. My goal is to see if the predictive ability of the model/variables has changed across cohorts. Given that these are observational data, I think the parameter estimates should only be interpreted as the predicted difference in the response of two individuals that differ by one unit on the regressor in question and that have the same value on all other regressors. I am not making inferences about changes or causality. Thank you. $\endgroup$
    – Grad student
    Commented Feb 14, 2014 at 20:23

1 Answer 1

Reset to default
2
$\begingroup$

I'd strongly discourage including those intermediate outcomes, for the sake of interpreting the parameters. Let's translate "predicted difference in the response of two individuals that differ by one unit on the regressor in question and that have the same value on all other regressors" into the terms of the study question: if you include the intermediate outcomes in the model, you're only comparing students who delayed enrollment to others who delayed enrollment, and comparing students who went to similar colleges to each other. If students with lower high school GPA are also more likely to delay college enrollment, and students who go to private colleges have higher GPAs, a model that includes both variables will only be useful for predicting a student finishing college based on their GPA if you also already know what kind of college they went to. Is that the prediction you need?

Improve this answer
$\endgroup$
2
  • $\begingroup$ I believe I understand you're comment, but I'm not sure the interpretation is the issue. I thought the comparison would be those that delayed enrollment to those that did not delay, all else being equal (including college characteristics). The college characteristics I have include things like public/private, enrollment size, and funding. Does that help to clarify? I've added more detail to the original post. $\endgroup$
    – Grad student
    Commented Feb 16, 2014 at 1:01
  • $\begingroup$ That's right, it's the "all else being equal" that causes the problem, if you know that in fact there is a correlation between the predictors. Then the coefficients only estimate the "independent" effect of, say, GPA adjusted for college characteristics. But since GPA influences what college a student goes to, adjusting for college will mask the effect of GPA on the final outcome. $\endgroup$
    – vafisher
    Commented Feb 17, 2014 at 14:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.