When is it acceptable to probe an interaction effect between categorical variables? (Regression vs. anova)

Question

This question seems to be an overly simple one but I would still appreciate an informed response.

I am testing the interaction effect between a categorical variable (5 levels) and a binary variable. Outcome variable is continuous.

I have completed by regression and anova to test for a possible interaction effect. Results of regression indicates a significant interaction effect between two groups (EX: class 1 and 2).

In short, the regression output suggests a significant interaction effect (Class=3 * BIN=1) and the anova output suggests that the SSQ of the interaction term is non-significant (CLASS:BIN).

(1) Does the significant interaction term (Class=3 * BIN=1) in the regression output justify "probing" of the significant interaction effect?

(2) What is the significance of the (non-significant) interaction term (CLASS:BIN) in anova?

(3) Say the moderator and DV are flipped (categorical by continuous predicting binary) so you are now running a binary logistic regression. P-value for the contrast of interaction effect for 2 dummy classes is still significant. Does it now make sense to probe the interaction solely within a regression framework? Would you not if the anova output is still saying the whole interaction term is NOT significant?

Regression output:

 Coef              S.E.   t       Pr(>|t|)
 Intercept         0.6533 0.2402  2.72 0.0068  
 Class=2          -0.6955 0.2997 -2.32 0.0208  
 Class=3          -1.0229 0.2617 -3.91 0.0001  
 Class=4          -0.8226 0.2891 -2.84 0.0047  
 Class=5          -0.6086 0.3250 -1.87 0.0619  
 BIN=1             -0.3671 0.3082 -1.19 0.2343  
 Class=2 * BIN=1  0.7025 0.3955  1.78 0.0765  
 Class=3 * BIN=1  0.8602 0.3561  2.42 0.0162 
 Class=4 * BIN=1  0.6066 0.3743  1.62 0.1058  
 Class=5 * BIN=1  0.5961 0.4285  1.39 0.1649

Anova Table:

            SSQ     df1   df2     F value  Pr(>F)    eta2    partial.eta2
CLASS       19.04070  4  621.7598 4.0077   0.00322  0.04562   0.04726
BIN         7.11855   1 1014.6402 6.2523   0.01256  0.01706   0.01821
CLASS:BIN   7.40183   4  479.6875 1.3864   0.23751  0.01773   0.01892
Residual    383.82114  NA    NA      NA      NA      NA           NA

Answer 1

1) yes, it does justify probing the significance of the CLASS:BIN interaction with an ANOVA. However it is not entirely clear to me what does the term Class=3*BIN=1 represent in your table. Calling Y the outcome, is it the difference Y(Class=1,BIN=1)-Y(Class=3,BIN=1)?

2) the ANOVA performs instead a model comparison, asking whether the addition of the interaction term (CLASS:BIN) explained a large enough amount of variance in the data, with respect to the number of degrees of freedom it added (4)

based on the significance of the term Class=3*BIN=1 you cannot conclude that the effect of BIN on Class=1 and Class=3 differs. Indeed 2) suggests it probably does not. To test whether there is a difference you should build contrasts comparing the effect of BIN on each of the Classes. Then test for the significance of each contrast, keeping into account you are performing multiple tests and that you should correct for that. To help you more I should know exactly what do the coefficients in your table represent.