3
$\begingroup$

I'd like to start off by saying I am very much a beginner to statistical analyses, so I might be missing some key ideas here.

Context: My study involves an experimental group and control group that received different exercises. Within the groups, some participants identified as male, some as female. I measured participant satisfaction in completing the exercises.

Question: I want to know if the experiences of female students within the two groups differed in terms of how satisfied they were.

Tests run: (1) A two-way ANOVA with female/male + group (experimental or control) as my two independent variables, and satisfaction measure (based on survey of scale questions) as dependent variable. (2) I conducted t-test of just female students and compared their motivation ratings in experimental vs. control group, and another t-test of just male students and compared their satisfaction ratings in experimental vs. control group.

Results: (1) The ANOVA showed non-significant effects (p > 0.05) for both gender, and gender * group interaction. (2) The t-tests showed that female students had significantly higher satisfaction in experimental group, and male students had no difference in satisfaction among the two groups.

Question: Doesn't the results of (2) suggest an interaction effect? Unless I'm misunderstanding what ANOVA results mean completely.. I checked all assumptions of normality and equality of variances before running each of the tests using SPSS.

$\endgroup$
2
  • $\begingroup$ How many participants were in each group for your IVs? Also, what were the p-values when you ran the ANOVA? $\endgroup$
    – Nate
    Commented Dec 14, 2019 at 22:38
  • $\begingroup$ Yes, please give us the ANOVA and t-test results. It seems that the group effect was significant? But that was mainly driven by females? $\endgroup$
    – stweb
    Commented Jul 15, 2023 at 11:34

2 Answers 2

0
$\begingroup$

A better way to approach this analysis could be to:

  1. Fit the ANOVA model to test the overall significance and/or;
  2. Perform pairwise comparisons on the ANOVA model to test individual effects for significance while controlling the Family Wise Error Rate (FWER), e.g. Tukey's HSD test, Dunnett's test. Your t-test results apparently were not adjusted for multiple comparisons?

You might also find the following posts helpful:

$\endgroup$
0
$\begingroup$

The basic problem here is interpreting the main effect when there's a significant interaction. This is generally bad practice.

A secondary issue is comparing an ANOVA with a t-test on a much smaller sample size. The degrees of freedom in the models are very different.

Using the output of the ANOVA makes most sense here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.