6
$\begingroup$

I'm interested in the effect of X on Y and want to adjust for confounding variables in my regression model. If the model (regression, F-test) is not significant but the predictor of which I'm interested in is, could I still report that there is an association between X and Y? So I just wanted to adjust for confounding variables but my interest is the relation between X and Y. Thank you.

Lauren

$\endgroup$
  • 1
    $\begingroup$ The title does not exactly match the content. Anyway, the model not being significant is an indication you might want to consider another model. I am not sure about the answer to your main question, though. If you rely on the model, you can surely report the association between X and Y. However, should you rely on an insignificant model? Hmm... $\endgroup$ – Richard Hardy Aug 15 '15 at 9:49
  • $\begingroup$ I'm primary interested in x - y, but I have to adjust for confounding variables. Apparently all these confounding variables in this model are not significantly related to y and that's why the overall regression model (F) is not significant. But my main focus x - y is and this predictor is still significant although adjusted for confounders. Is this more clear? Thank you for your advise. $\endgroup$ – Lauren Mcarthur Aug 15 '15 at 10:04
  • $\begingroup$ Your question is clear, I get it from the original post (but thanks for the extra explanation). However, I am not sure about the answer. Let's hope someone else will join and do better. $\endgroup$ – Richard Hardy Aug 15 '15 at 10:40
5
$\begingroup$

One purpose of regression is to control for the effects of covariates. This question is predicated on the (correct) understanding that this purpose should not be confused with testing the significance of those covariates.


In a linear multiple regression model

$$\mathbb{E}(y) = \alpha + \beta_1 x_1 + \cdots + \beta_k x_k,$$

the $F$-test compares the null hypothesis

$$H_0: \beta_1 = \beta_2 = \cdots = \beta_k = 0$$

to the alternative

$$H_1: \beta_j \ne 0\text{ for at least one }j.$$

In your case, you're not interested in this hypothesis because most of those coefficients are associated with covariates. Letting $j$ be the index of the single predictor in which you are interested and $n$ be the amount of data, your test should be based on comparing

$$H_0: \beta_j = 0$$

to

$$H_1: \beta_j \ne 0.$$

This is usually done with a t-test in which the estimate $\hat \beta_j$ is divided by its standard error $se(\hat\beta_j)$ and the resulting t-statistic is referred to the Student t distribution with $n-k-1$ degrees of freedom. If you consider that result to be significant, then you will reject this null hypothesis (rather than the omnibus null hypothesis of the F test) and conclude that after controlling for all covariates, variable $x_j$ was found to be significantly associated with $y$.


Additional considerations

Note that if you intended to conduct several such tests separately, involving several variables, then this procedure would no longer be correct for any one of them. Context matters! You would need first to perform a test to see whether any of that set of variables is significant. The usual procedure is an F test based on the "extra sum of squares" associated with the variables of interest. In the case of a single variable, this F test is mathematically equivalent to the Student t test.

More subtly, note that what matters is the number of tests you planned to make before seeing the data. If first you examined the data and then based on that examination you selected $x_j$ as the sole variable of interest, then you would somehow have to figure out how to account for the additional information you used in order to narrow the model down to this single variable. You might, for instance, attempt (as honestly as possible) to enumerate all the variables you could possibly ever have been interested in testing, then treat them as a group as just described.


Reference

Montgomery, Peck, and Vining, Introduction to Linear Regression Analysis. Fifth Edition, 2012. John Wiley & Sons. Section 3.3.

$\endgroup$
  • $\begingroup$ Great answer, as always! But does it imply that the individual significance of a coefficient of interest can be trusted if the whole model is insignificant (and thus the whole model "cannot be trusted")? I suppose if the whole model is poor in one sense or another, conclusions based on the model (or some part of it) should be taken with extra care or avoided at all. I know I am not being precise here, but I did not quite get the message and I intend to provoke it here. $\endgroup$ – Richard Hardy Aug 17 '15 at 20:06
  • $\begingroup$ @Richard I'm sorry for not being clearer. There is only one relevant hypothesis here--and it's not the one being evaluated with the F test, it's the one evaluated with the t test. As always, everything depends on the statistical assumptions being (at least approximately) correct: the model is fully specified, the errors are IID normal, etc., etc. I would depart from some textbook recommendations--Montgomery et al included--in maintaining that if you had a prior reason for including covariates, then you're justified keeping them in the model, even when they're "insignificant." $\endgroup$ – whuber Aug 17 '15 at 20:45
  • 1
    $\begingroup$ Thanks! I can buy that if the statistical assumptions are at least approximately correct and the only visible fault of the model is it's insignificance, then that fault can be ignored if the goal is to test for statistical significance of one of the mdel's coefficients. $\endgroup$ – Richard Hardy Aug 18 '15 at 16:27
  • 1
    $\begingroup$ Thank you for this valuable comment. It always seemed logic to me that if you are interested in X-Y and control for confounding factors (based on literature) you are primarily interested in X-Y, not the complete model. It is finally clear to me :). $\endgroup$ – Lauren Mcarthur Aug 19 '15 at 14:37
0
$\begingroup$

Null hypothesis under the Overall F-test is the following:

 H0: β1 = β2 = ... = βm= 0 
 Ha: At least one of the slope parameters is not equal to 0.

Looking at the alternative hypothesis (Ha), I doubt you can say anything about individual coefficients once you fail to reject the Null.

Having said that, failing to reject the NULL only means that the relationship between X and Y cannot be explained by the given model (in this case linear). You might try another specification or a non-linear specification between X and Y.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.