0
$\begingroup$

I did a general linear model with two independent variables (math (a continuous variable) and group (a categorical variable with 2 categories)) in SPSS. I have included the interaction effects of group and math in the model. The results looked as follows. My question is how to interpret the effect for math? The results of the F-test says it's significant (p = 0.008), but the results of the t-test says it is not significant (p = 0.111). Please help.enter image description here

$\endgroup$
0
$\begingroup$

The interpretation of the slope for a continuous variable when there is an interaction with a binary variable is as the effect of the continuous variable at the reference level of the binary variable. In this case, we would interpret the coefficient on Math (.248) as: "For one a one-unit increase in Math score, the outcome is expected to increase by .248 points for those in group 2."

The reason the F-test and t-test for Math yield different results is that they test different things. The F-test tests whether Math has an effect on the outcome averaging over group, and the t-test tests whether Math has an effect on the outcome for those in group 2. The first test is for a "marginal" effect, and the second test is for a "conditional" effect.

You might interpret the results by saying that it's not clear whether there is an effect of Math for those in group 2, but there is an effect of Math in the population at large (i.e., the population from which the sample was drawn). As for the question of whether there is an effect of Math for those in group 1, your analysis as you've performed it doesn't allow you to directly answer that question without some follow-up tests.


Follow-up: The F-test tests whether, in a hypothetical population with equal proportions of units in groups 1 and 2, there is an effect of Math, or whether the average of the group-specific slopes is different from zero. If you parameterize the regression in the standard way as follows: $$Y=\beta_0+\beta_M X_M +\beta_G G_2 +\beta_{MG}X_M G_2 + e$$ where $G_2$ is an indicator for being in group 2, $\beta_M$ is the slope of $X_m$ for group 2 (i.e., the reference group), and $\beta_M + \beta_{MG}$ is the slope of $X_m$ for group 1. The Wald t-test (in the regression output) tests whether $\beta_M=0$. The ANOVA F-test tests whether the average of the slopes in groups 1 and 2 is zero (i.e., $\frac{1}{2}(\beta_M + \beta_M + \beta_{MG}) = 0$). This is kind of a contrived hypothesis that doesn't map well onto a usable estimand because it refers to a hypothetical population where the probabilities of being in groups 1 and 2 are $.5$, which may not be realistic in practice. In a balanced sample where groups 1 and 2 have the same number of units, this test makes more sense.

Typically it makes more sense to compute the average marginal effect of $X_M$, for which you can use a different parameterization, as follows: $$Y=\beta_0+\beta_M^* X_M +\beta_G^* (G_2 - \bar{G}_2) +\beta_{MG}^*X_M (G_2 - \bar{G}_2) + e$$

where $\bar{G}_2$ is the proportion of units in group 2 in the sample (or population). Now, the interpretation of $\beta_M^*$ is the average effect of $X_M$ averaging across everyone in the population. This is a more usable interpretation. The other coefficients and tests (other than the intercept) will remain the same regardless of which parameterization you choose. If you replace $\bar{G}_2$ with $.5$, then $\beta_M^*$ will again be the average of the group-specific slopes, and the t-test on $\beta_M^*$ will be equivalent to the ANOVA F-test.

$\endgroup$
  • $\begingroup$ The model is $Y = \beta_0 + \beta_1X_1 + \beta_2 X_2 + \beta_3 X_1X_2 +e$. According to your second paragraph, could you give the two null hypotheses for F-test and t-test? It is interesting. $\endgroup$ – user158565 Jul 2 at 4:04
  • $\begingroup$ I responded above. $\endgroup$ – Noah Jul 2 at 4:37
  • $\begingroup$ I found the null hypothesis for F-test. Where is the null hypothesis for t-test? $\endgroup$ – user158565 Jul 2 at 5:35
  • $\begingroup$ I restructured my answer, so take a look at it again. Hopefully it's clearer. $\endgroup$ – Noah Jul 2 at 6:14
  • $\begingroup$ The type 3 F-test compares the two models:$$Y=\beta_0+\beta_M X_M +\beta_G G_2 +\beta_{MG}X_M G_2 + e$$ and $$Y=\beta_0+ 0 X_M + \beta_GG_2 +\beta_{MG}X_M G_2 + e$$. So the null hypothesis is $\beta_M = 0$. $\endgroup$ – user158565 Jul 4 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.