7
$\begingroup$

I have a question (simpler than my previous post today I hope!), which is probably very stupid as nobody has never asked it before.

Lets say I am trying to explain the effect of 3 variables (A, B and C) on a dependent one (Y). Biologically speaking, A and B should really have an effect on Y. So I am testing:

Y ~ A + B + C

But when I use a model selection method (whatever the method is), the 'best' model, the one that fits the data the best, drops A. So I end up with:

Y ~ B + C

What can I say about A then?

Can I cite something to justify the dropping (the F statistics, something about the AIC/BIC, etc)?

OR

If I need to show that A has no effect, do I need to use the full model anyway?

Improve this question
$\endgroup$
8
  • $\begingroup$ AIC, BIC, DIC, Likelihood Ratio, cross-validation, etc... are tools that you can use depending on the features of the model you are interested on. If they indicate that A can be dropped, then you could use the Occam's razor to defend the model $B+C$. The other post is interestig, but a bit long. Shorter questions seem to be more successful, you know, the popular TL;DR. $\endgroup$
    – user10525
    May 1, 2012 at 19:57
  • 1
    $\begingroup$ Thanks. And I agree for the tl;dr thing, but I like to be clear, and thats also a very specific case, so I had to fully describe it. Thats the complexity of that dataset that makes it challenging to analyze. $\endgroup$
    – Joe
    May 1, 2012 at 20:47
  • $\begingroup$ And for your answer here: it's not really my question though=> I want to know what to say about the dropped variable. I need a way to prove that it had no effect. $\endgroup$
    – Joe
    May 1, 2012 at 20:49
  • 3
    $\begingroup$ Don't confuse dropping a variable with having no effect, Joe! For example, let y ~ a+d+e, b=a/2+2d+e/2, c=a/2-d+e/2, with d, e, (and the implicit error) all small compared to the variation in a. Then a may have a profound effect on y, but (due to its association with b and c) is unnecessary in the model (and won't even be significant in the full model y ~ a+b+c); the best model would be y ~ b+c (because this equals a+d+e). $\endgroup$
    – whuber ♦
    May 1, 2012 at 21:13
  • 1
    $\begingroup$ @whuber That's very well-put. Why not make it an answer? $\endgroup$
    – Matt Parker
    May 1, 2012 at 22:28

1 Answer 1

Reset to default
5
$\begingroup$

It depends on your aim. Since A "really should" have an effect on Y but appears not to, I would definitely include the full model. As Procrastinator suggests, model selection might be useful to propose a parsimonious new model - if that's your objective. But if your objective is to estimate the relationships between your predictors and Y, then I don't think you need a model selection step at all - just present your full model results.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Ok thanks. I guess I got confused about the role of model selection, which is probably not appropriate in that case. I just thought that 'fitting the data better' was better, but actually it's not here! $\endgroup$
    – Joe
    May 1, 2012 at 21:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.