3
$\begingroup$

I am identifying risk factors for children snoring among several predictors with generalized linear model. With backward selection, age and sex do not appear to be significant so I removed them from the final model.

  1. Should I still keep them as possible confounders?
  2. If I keep them, what is the interpretation?
  3. Could there be confounders in significant predictors? The significant predictors are BMI, an environmental factor, several parental and childhood diseases, and two of their interactions.
Improve this question
$\endgroup$
6
$\begingroup$

You should not use backwards selection. There are numerous problems with it, including: it is extremely unlikely to pick the 'right' variables, the p-values will be invalid, and the model's out of sample predictive performance will be worse. (For more information, see my answer here: Algorithms for automatic model selection.)

In addition, whether or not a variable is significant isn't quite what you're after. You state that you want to identify risk factors. There are at least two interpretations of that. You might want to identify variables that are associated with / predictive of the response, whether they are causally related or not. These would be variables that are easier to assess, and would thus serve as screeners to alert doctors that a given patient should receive more scrutiny. The other interpretation is variables that have a causal impact on the response. The idea here is that these variables should be targets for interventions to prevent or minimize the response. These two perspectives sound similar, but they are not the same (see: 1, 2, and/or 3 (pdf)).

Your discussion of confounding implies you have the latter in mind. It is possible to achieve unbiased causal estimates with observational data by controlling for confounders, but that is more difficult than most people seem to think. At any rate, what it requires is controlling for all the causally relevant variables (and not controlling for the wrong variables), not controlling for all significant variables. A cause might not be significant in your model for any number of reasons, and inappropriate variables (e.g., mediators or common effects) might be significant despite not being the variables you want.

  1. If you have reason to include any variables (such as age and sex in this case), you should include them whether they are significant or not.
  2. The interpretation of those variables is the same as ever: a one-unit increase is associated with $\hat{\beta_j}$ increase in $Y$ (presumably the log of the odds of snoring, here). In addition, you could say their association is ambiguous in that you have insufficient evidence to be sure of the sign of the association to your chosen level of confidence. More generally however, the interpretation of variables included as possible confounders is typically of secondary interest; you are after the interpretation of candidate risk factors after controlling for confounders. In essence, they are almost nuisance variables.
  3. Typically, counfounders and predictors have a different ontological status in the model. That said, you certainly could be mistaken about what the status of a variable is and some of the things you had been thinking of as potential predictors are, in fact, confounders.
Improve this answer
$\endgroup$
  • $\begingroup$ Very informative. To clarify, my model is supposed to be an explanatory one while you said "identify variables that are associated with / predictive". My understanding is that a predictive model just includes causal independent variables, and an explanatory model can include all variables that seem relevant, that is, to define the association. Your definition looks obscure to me. $\endgroup$ – junjianliu Sep 26 '20 at 3:29
  • $\begingroup$ Besides, variables from the original survey are too many and somehow redundant and it is hard for me to do variable selection. I do have one potential risk factor and several known risk factors in the model, should I treat the known ones as confounders? Anyway I will include age and sex afterall since it is an epid study... $\endgroup$ – junjianliu Sep 26 '20 at 3:30
  • $\begingroup$ @junjianliu, read the links. Explanatory means causal; a predictive model exists just to allow you (in the future) to estimate y-hats for patients where y is unknown. I'm sympathetic w/ the situation that you have a difficult set of variables to choose amongst. Nonetheless, backwards selection is invalid. Variable selection must be done before looking at the data. It sounds like the point of your study is to assess the causal status of a particular potential risk factor, while controlling for other variables. You should look into methods for causal inference. $\endgroup$ – gung - Reinstate Monica Sep 26 '20 at 8:32
  • 1
    $\begingroup$ A small addition to point 2 - it's worth noting that the coefficients for the confounding variables can't be interpreted causally in the same way as the primary risk factor. This is because these coefficients will be adjusted for the primary risk factor - which can be inappropriate as the primary risk factor often lies on the causal path between the confounder and outcome. The effects of these confounders are also subject to their own confounding which is rarely considered $\endgroup$ – Lachlan Sep 27 '20 at 3:05
  • $\begingroup$ Excellent points, @Lachlan. $\endgroup$ – gung - Reinstate Monica Sep 27 '20 at 4:33
4
$\begingroup$

With backward selection

Don't do this. Variable selection a la these backward/forward methods is not a good way to determine which variables should be included in the model. This has been discussed many times on this forum.

As to your questions

  1. Only if you think your estimates require age and sex adjustment. That will have to be determined via domain knowledge. Non-significant variables need not be removed from the model. They could potentially have large but uncertain effects, in which case I think they should remain in the model.

  2. They offer the same interpretation as they would had they been significant. We just can't reject the null hypothesis that the effects are 0.

  3. Yes, confounding could still be present even though you have significant predictors.

Improve this answer
$\endgroup$
  • $\begingroup$ Thank you! Could you please refer me some links of valid variable selection methods? $\endgroup$ – junjianliu Sep 26 '20 at 3:31
  • $\begingroup$ @junjianliu this article addresses these questions in the epidemiology context $\endgroup$ – Lachlan Sep 29 '20 at 2:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.