Testing significant interaction effect across 4 different scenarios: what types of alternative indices are there?

Question

There appears to be a few possible alternatives to determining whether there is a significant interaction effect.

It is not clear to me whether testing the statistical significance of an interaction term works the same way for different types of scenarios:

(1) 2 WAY ANOVA: Categorical by Binary (where A= 4 level factor, leading to 3 interaction terms being tested)

• It seems that F-value for specific interaction term is the most widely used.

• Are the following acceptable alternative indices? incremental tests using chi-square or F-value change with and without the interaction term

(2) Continuous by Continuous

• Significance of the overall model (Y=A+B+AB, using overall F-value) and the significant p-value for AB

(4) Continuous by Categorical - Anova would still generate an F-value for (A, B, A*B). - Are there alternative indices for this scenario when A is a factor (4 levels), B is binary?

(5) Categorical by Categorical by Continuous (3-way interaction)

Suggestions for any or all of the above (keeping in mind R is the platform) would be much appreciated.

Answer 1

I recommend thinking about ANOVA in terms of linear regression. You will find out that all tests of factors and interaction terms are exactly the same test – comparison of model and submodel.

Nominal factor in ANOVA can be represented as a set of dummy variables (example: dummy coding ). To test significance of the main effect, compare your model with a submodel without the dummy variables of the particular factor. The F statistic is used:

$$ F = \frac{\frac{S^2_{submodel} - S^2_{model}}{df_{submodel}- df_{model}}}{\frac{S^2_{model}}{ df_{model}}} \sim F_{df_{submodel}- df_{model}, df_{model}} $$ where $S^2$ is residual sum of squares, $df$ are degrees of freedom.

If your ANOVA contains interaction, add interaction term in regression equation. Interaction term is simply a product of interacting variables. Interaction of continuous or alternative variable $X_1$ and $X_2$ is variable $ X_1 \cdot X_2$. If $X_1$ and/or $X_2$ are nominal factors create interaction term for each pair of their dunny variables (e.g. if you have 4*6 ANOVA design with interaction, you code $X_1$ with 3 dummy variables, $X_2$ with 5 dummy variables, and there are also 15 interaction variables).

The test of significance of interaction is again the comparison of your model with a submodel without the interaction variables. Again, you obtain $F$ statistic. It is the same statistic regardless the fact you test continuous, nominal, interaction…

---

You mentioned $\chi^2$ test. It does the same but assumes that the amount of error variance is known, which is usually not true. I don’t recommend.

In regression you can also find so called Wald statistic with t distribution. Wald statistic squared is exactly the same as our F statistic. It also produces the same p-value.