I am unsure of how to establish (or interpret) the independence condition of the chi-squared test in (for instance) the following situation.

Consider this 2x2 contingency table representing the number of 'statements' that are either negative or positive, by members from the genders 'male' and 'female'. None are members from both gender; however, some statements are made by the same person.

        |Negative                   Positive                    |Marginal Row Totals
Male    |7683   (7660.06)   [0.07]  2983   (3005.94)   [0.18]   |10666
Female  |1259   (1281.94)   [0.41]  526    (503.06)    [1.05]   |1785
        |8942                       3509                        |12451    (Grand Total)

This yields $\chi^2=1.7008$, $p=.192186$ (not significant). I have more tables like this, with different independent and dependent variables, but the same idea. Some of them are significant.

  • There are 150 subjects in total, and each typically make a number of 'negative' statements and a number of 'positive' statements, in a perceptual evaluation experiment where they are typically describing 20 to 40 audio fragments.
  • They can make any number of statements about any fragment, but here there are between 0 and 10, median 3.
  • There are about 180 fragments in total, i.e. each fragment is only assessed by a fraction of the subjects.

I therefore assume the following assumptions have been met:

  • simple random sample: hard to test (?), but the idea is that the subjects are representative of the population I'm examining
  • sufficient sample size (whole table): this looks bigger than any other contingency table I've seen
  • expected cell count: more than 5 or 10 in each cell

The one I am not so certain about is independence, as some of the statements are made by the same person, and some of the statements describe the same fragment. I wonder if this is mitigated by the huge numbers.

Of course, independence is exactly what this test is supposed to assess, hence the name, though perhaps my understanding of these different meanings of 'independence' is the problem.

McNemar's test seems inappropriate for this data, as there is no repeated measures or 'before/after' character to this.

I would appreciate any advice on the exact meaning of this 'independence', its effect, and how it can be justified (in case it's fine) or addressed (in case this data needs a different statistical technique).

  • 1
    $\begingroup$ Do the subjects make single statements about multiple fragments, multiple statements about single fragments, or multiple - multiple? (Your data are not independent in the sense required by the chi-squared test.) $\endgroup$ – gung - Reinstate Monica Jan 3 '17 at 20:47
  • $\begingroup$ Multiple statements (0-10 per fragment) about multiple fragments (20-40 per fragment). Any statistical approach that's more suitable? And what's the likely effect of deviating from this condition, e.g. Type I / Type II errors? $\endgroup$ – BrechtDeMan Jan 3 '17 at 20:51
  • 1
    $\begingroup$ Increased type I errors--you are acting as though you have more data than you really do. Are all statements constrained to be either positive or negative (ie, are there neutral statements)? Are subjects constrained to make only so many statements, or could they always have made 1 more? Are you just interested in if men are more positive than women? $\endgroup$ – gung - Reinstate Monica Jan 3 '17 at 21:03
  • $\begingroup$ There are <1% neutral/'unknown' statements, not included here. No constraints on the number of statements. Essentially, yes I just want to know whether women make more or less negative statements than men about this type of stimuli. $\endgroup$ – BrechtDeMan Jan 3 '17 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.