1
$\begingroup$

I ran a textbook example in an Excel sheet - and replicated the answer. However when I retain the ratio, but double the sample size my result is not significant.

This strikes me as illogical. Please can someone explain where I have gone wrong, or why a fixed ratio does not give same chi-square result.

Test used X2 = (Abs(O-E)-0.5)squared / E
df= 1 (2 categories)
Test 1 (text book example- significant diff)
Females 4
Males 12
Total 16
Expected Females 8
Expected Males 8
Yates corrected X2 = 3.06

Test 2 - same ratio, diff sample size
Females 8
Males 24
Total 32
Expected Females 16 
Expected Males 16
Yates corrected X2 = 7.03
Improve this question
$\endgroup$
  • $\begingroup$ degree of freedom is not 1.It is 16 -1= 15 and 32 -1 = 31for test -2. Please correct it. $\endgroup$ – Subhash C. Davar Nov 14 '17 at 10:33
  • $\begingroup$ @subhashc.davar Sorry, but that is just confused at a very elementary level. $\endgroup$ – Nick Cox Nov 14 '17 at 11:58
2
$\begingroup$

There are two questions here, specific and general.

In general, the logic of significance tests certainly does include consideration of sample size.

In my favourite software I get (without Yates' correction)

observed frequencies from keyboard; expected frequencies equal

         Pearson chi2(1) =   4.0000   Pr =  0.046
likelihood-ratio chi2(1) =   4.1860   Pr =  0.041

  +-------------------------------------------+
  | observed   expected   obs - exp   Pearson |
  |-------------------------------------------|
  |        4      8.000      -4.000    -1.414 |
  |       12      8.000       4.000     1.414 |
  +-------------------------------------------+

observed frequencies from keyboard; expected frequencies equal

         Pearson chi2(1) =   8.0000   Pr =  0.005
likelihood-ratio chi2(1) =   8.3720   Pr =  0.004

  +-------------------------------------------+
  | observed   expected   obs - exp   Pearson |
  |-------------------------------------------|
  |        8     16.000      -8.000    -2.000 |
  |       24     16.000       8.000     2.000 |
  +-------------------------------------------+

Specifically, it is hard to see why you describe the results as you do. Your output doesn't include observed significance levels at all, but either test yields significance at conventional level (all P-values are $\le$ 0.046, compared with the conventional level of 0.05 = 5%). The second has higher chi-square and thus for the same number of degrees of freedom a lower P-value (in ordinary language, a more significant result).

On the general point, take a different example: You're tossing a coin and get 6/10 heads. Is the coin fair (with probability 0.5 for heads, 0.5 for tails)? Do you regard possible proportions 60/100, 600/1000, ... 6 billion/10 billion, ... in exactly the same way? 6/10 could easily be a fluke, in ordinary language, but at some point as you get higher chi-square and lower P-values, you should change your mind.

The use of Yates' correction here is a side-issue except insofar as it affects threshold significance levels.

Improve this answer
$\endgroup$
  • $\begingroup$ Excel tells me that the X2 value is greater and the probability lower- as per Nick Cox answer. HOWEVER the book had me look up an Appendix and informed me that my X2 value had to be LESS than a threshold value for p < 0.5 OR p < 0.01 and that leaves me most confused. I am going to trust Excel (!) and ignore the black box sensation that gives me! $\endgroup$ – m4sterbunny May 26 '17 at 13:03
  • 1
    $\begingroup$ Unnamed and so inaccessible book, so I can't possibly comment reliably. I agree that chi-square should be larger than stated threshold values to qualify as more significant. $\endgroup$ – Nick Cox May 26 '17 at 13:07
  • $\begingroup$ @m4sterbunny It might be worth quoting the book, with some context and asking about it (in another question) $\endgroup$ – Glen_b May 26 '17 at 17:13
  • $\begingroup$ Degree of freedom here should be calculated as n-1 and not as number of categories - 1. $\endgroup$ – Subhash C. Davar Nov 14 '17 at 10:38
  • $\begingroup$ @subhashc.davar Your suggestion is extraordinary. Care to flesh it out? $\endgroup$ – Nick Cox Nov 14 '17 at 10:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.