0
$\begingroup$

I'm reading a paper that compares two vaccine types. THe people who got vaccine 1 are of lower SES and have more chronic conditions compared to those who got vaccine 2. Multivariate logistic reg was used to compare the vaccines on mortality, emergency room visits, diagnosis and a couple other outcomes (no adjustment for multiple comparisons). In the logistic model they controlled for SES, chronic health conditions, sex and age. Vaccine 2 was found to be significantly protective for all outcomes compared to vaccine 1. How do I know that the statistical control was good enough to account for the group differences (SES and chronic conditions)? Seems like the deck was stacked against vaccine 1.

$\endgroup$
2
  • 2
    $\begingroup$ In principle such controls can work, but their success depends on the details. Please add a web link to the paper in your question so that those details can be examined. $\endgroup$
    – EdM
    Commented Oct 4, 2018 at 14:27
  • 1
    $\begingroup$ This is a difficult question that would require significant context and field specific expertise to reasonably answer. $\endgroup$ Commented Oct 4, 2018 at 15:07

1 Answer 1

0
$\begingroup$

You are right to question the inclusion of control variables, but this issue is a current debate in statistics. Rubin (1974) along with his protege Gelman (2013) feel that including as many control variables as possible is helpful for model estimation. Gelman in particular seems to believe that including more control variables increases information gained by having more joint probability distributions. However, Pearl (2000) along with Spector and Brannick (2010) and others believe that including such control variables without a strong justification may lead to contamination of outcomes. For example, the model in the paper you mention implies that SES has an effect on mortality and ER visits, or x->y and x->z, but perhaps SES only effects ER visits and not mortality, such that it is fair to say x->z but x->y would be an urban legend (Spector & Brannick, 2010). I like the prospect of gaining ifnormation by creating more joint probability distributions, a la Gelman, but the potential contamination la la Pearl et al gives pause. I would question any model that includes control variables without justification.

$\endgroup$
2
  • 1
    $\begingroup$ My concern is more with the adequacy of statistical control via a multivariate model. The groups are not similar on several important variables but by putting those variables into a model we can now feel confident that any differences we see is based on the type of vaccine used and not that the groups were different to begin with? $\endgroup$
    – MHuck
    Commented Oct 4, 2018 at 15:36
  • $\begingroup$ As I said, such confidence comes from two places: the justification of inclusion of the variables in the model as well as the model itself. Including control variables to be confident whether "differences we see is based on the type of vaccine used" without justification is an urban legend. There must be justification for including control variables. I guess I side with Pearl and Spector & Brannick. $\endgroup$ Commented Oct 4, 2018 at 17:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.