7
$\begingroup$

My topic is faking emotional intelligence. I have a between-subjects design with three conditions. People were asked to complete the emotional intelligence survey in either of these conditions:

  1. as a job applicant
  2. being self-critical
  3. no instructions (considered "honest condition").

An ANOVA did indeed show that people do fake: those in the job applicant stage had significantly higher EI scores than those in the honest condition; this is how I have defined faking.

I am most interested in seeing whether there are gender differences in faking EI. From a line graph, I can see that women fake more than men (women have much higher scores in condition 1 vs condition 3; men have a smaller gap between scores in condition 1 vs 3). From the following three analyses, it appears that women do fake more than men: 1) As mentioned, those in the job applicant condition have significantly higher scores than those in the honest condition 2) Filtering for only the job applicant condition, an ANOVA reveals that women score significantly higher on EI than men 3) Filtering for only the honest condition, an ANOVA shows that there are no significant differences in EI scores between men and women

BUT, the problem is I'm not sure if what I described above is essentially like conducting a two-way ANOVA with gender and condition as IVs and the EI scores as the DV. I had thought of this initially and ran this ANOVA but failed to get a significant gender X condition interaction (filtered just for conditions 1 and 3).

My question is: based on the two ways I approached my question about gender differences, which is correct? My three-step analysis which infers that women do fake more? Or the two-way ANOVA which shows no significant interaction result, suggesting no gender differences in faking?

$\endgroup$
  • 1
    $\begingroup$ You define three conditions & then talk about an undefined fourth. What's with that? In any case, it sounds like reading this paper might help you: Gelman (2006), The difference between 'significant' and 'not significant' is not itself statistically significant, The American Statistician, 60, 4 $\endgroup$ – Scortchi - Reinstate Monica Aug 12 '13 at 12:40
  • $\begingroup$ Sorry, for this analysis, I was only looking at three conditions. Anytime I mention condition 4, I really mean Condition 3. Thanks for catching that! Thank you also for the recommended paper and for your helpful answer below. $\endgroup$ – Jade Aug 12 '13 at 16:13
  • $\begingroup$ I fixed the numbering in the question. (You can edit too, clicking on 'edit' below the question text.) Please mark the answer as accepted if all's clear now, or don't if it isn't. Also consider reading Abelson (1995),Statistics as Principled Argument, which discusses this topic, among others, using examples drawn from experimental psychology. $\endgroup$ – Scortchi - Reinstate Monica Aug 12 '13 at 16:25
1
$\begingroup$

What you're trying to do here is estimate the difference between women & men in the difference between the job-applicant & no-instructions conditions in emotional intelligence score. Differences of differences are interaction effects, & must be analysed as such—it's the whole point of setting up a controlled experiment as you have done.

Consider:

Filtering for only the job applicant condition, an ANOVA reveals that women score significantly higher on EI than men (p=0.049)

Filtering for only the honest condition, an ANOVA shows that there are no significant differences in EI scores between men and women (p=0.051)

The two effects could be almost the same size.

NB No significant interaction does not suggest no differences in faking. It only means your experiment was unable to tell you at a given significance level whether women or men fake more. Look at the confidence interval for the interaction effect to assess whether there may or may not be any important difference between women & men in this respect.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.