Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite
1

I'm investigating associations between socioeconomic factors and dichotomous outcome. I use generalised linear models (GLM) with log link for Bernoulli family, i.e., modelling the prevalence ratio. At the epidemiology course of K.J. Rothman & E.Hatch we were told, that goodness of fit tests are designed to prediction models and in causal inference it is not important with model fit. I can not find any reference on that. Can anybody comment on this and suggest a reference? Thank you in advance!

share|improve this question

1 Answer

up vote 2 down vote

This is correct. In the epidemiology or social science, we would like to find the causal association between, say exposure and outcome. Then the most important thing is to identify the confounding factors, which need to be adjusted in your multivarate model settings. This does not necessarily mean to fit a model well, but only for adjustment purpose which makes ur estimate of interest unbaised due to other factors associated with outcome as well. For example, if we want to study the assocaiton between lung cancer and heavy drinking, then smoking status has to be adjusted as a confounding variable. Because smoking has been recognized as a risk factor of lung cancer. Therefore the heavy drinking status is confounded to the smoking, probably because heavy drinkers usually a smoker too. This is usually the utmost important consideration behind medical research.

If your purpose is for prediction, then you dont need to think confounding at first place, and you can include interaction terms, 2 ways, three ways interactions as a model budiling procedure, testing goodness-of-fit, and etc. And algorithms such as forward/backward selection are valid to provide a good model.

If you purpose is to provide a valid measure of an effect, then those selections are not quite appropriate. Because even a variable is not significant in a model, it still might be kept in the model such as age and gender, which are always adjusted in those epidemilogy study. Also the interaction effect might or might not be of interest in the epidemiology study.

Chapter 6 in the book of "Logistic Regression A Self-learning Text" provides a detailed explanation of model building strategy for what you asked about.

Cheers.

share|improve this answer
4  
I don't understand this reasoning. It seems that if you have a bad fit, then you could get the causal association completely wrong. Hormesis is a good example that should be important to epidemiology: if you fit a linear dose-response curve to a hormetic effect, you not only wind up overlooking the hormesis, but you draw entirely the wrong conclusions about low-level doses. How, then, can one justify overlooking this possibility by not checking goodness of fit? Am I misunderstanding your claims? – whuber♦ Dec 1 '11 at 16:45
1  
I must say i totally disagree with this approach, you always want a good model fit, for it indicates that your "causal" variable has been entered in the right way, and that it is not merely picking up random correlarions between causal variables which haven't been observed. also a poor goodness of fit makes it very difficult to say "A causes B" when there is so much extra things going on with the part of the data which didn't fit. – probabilityislogic Jan 31 '12 at 10:34
1  
@whuber It's an identification issue. The following minimal example might help: A and B independently cause C, i.e. A -> C <- B. A does not cause B so the corresponding regression coef of B on A should, if we want to interpret as a causal effect, be 0. But models also conditioning on C will all fit better than those that do not and also give a non-zero coef. Nevertheless this will be 'causally' wrong because this coef is the association but isn't the causal effect. And yes, from the data we cannot confirm the "A -> C <- B" part that grounds this argument. – conjugateprior Mar 1 '12 at 9:58
1  
@Conjugate Thank you for the clarification! I'm still not understanding, though, how this consideration would justify accepting lack of fit in a model. It seems to me the correct message is that we should use an appropriate model in the first place and interpret lack of fit as evidence that our model provides inadequate explanatory power (and therefore could give completely wrong results). After all, if you don't care about lack of fit, then nothing can falsify your model. You're no longer doing science--you're into fairy-tale and ESP land. Or so it seems to me... – whuber♦ Mar 1 '12 at 15:28
1  
@whuber Following up on my tiny example: The idea is not quite that nothing can falsify the model but that nothing in the data set alone can distinguish between the alternatives A <- C -> B and A -> C <- B, despite the fact that you should and should not condition on C respectively. However, manipulating C and watching A and B will tell you which structure you have and therefore whether you should condition on C. But under all circumstances a model with C in it should look better than one without. – conjugateprior Mar 2 '12 at 16:03
show 6 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.