3
$\begingroup$

I conduct Chi-squared test for following table:

|______|Feature1|Feature2|              |______|Feature1|Feature2|
|Group1|   70   |   30   |  100 =>      |Group1|   65   |   35   |  100
|Group2|   60   |   40   |  100 =>      |Group2|   65   |   35   |  100
           130  |   70   |

$\chi^2$ = 50/65 + 50/35 = 2.197;

P($\chi^2$ = 2.197) $\approx$ 0.15, so there would be no relation between Groups 1 and 2, if I chosed critical value = 0.20. However in case critical value is 0.05, I can't make any assumptions.

So the question is I want to compute size of sample (group) that I am to have, if I want to wait till the difference between groups becomes significant ($P(\chi^2) \le0.05$).

There is a size of the effect I would like to be able to detect, but I can't really get the physical meaning of this term. Could you give me an insight on current table?

or

if I were to observe a larger samples, in which the proportions were exactly the same as in this sample, then how large would this have to be for my result to be significant. And, yes, I'm aware of the case that samples may have different behaviour, if they become larger, but still)

$\endgroup$
  • 5
    $\begingroup$ If there truly is no relation between the groups, then you might have to wait a very long time indeed to observe a significant test statistic! It would be dangerous to assume the true difference is the one in your sample, because it is uncertain. So: do you have any other information to share, such as the size of the effect you would like to be able to detect? $\endgroup$ – whuber Sep 29 '16 at 20:46
  • $\begingroup$ "P(χ2 = 2.197) ≈ 0.15" obviously isn't quite right, are you missing an inequality sign? $\endgroup$ – Silverfish Sep 30 '16 at 11:30
  • $\begingroup$ It isn't really clear to me what you're asking, could you edit your question to clarify? Is it something like "if I were to observe a larger samples, in which the proportions were exactly the same as in this sample, then how large would this have to be for my result to be significant?" (In that case, be aware, there's no reason to think that the proportions you have observed in the small sample actually reflect the proportions in the population, nor that they will be found again in a larger sample!) $\endgroup$ – Silverfish Sep 30 '16 at 11:32
0
$\begingroup$
#all code given is in R, using the 'pwr' pachage.

What you are asking falls under power analysis. Power Analysis can help determine the sample size requirements to detect a specific effect size with a specific degree of confidence. If the proportion of observations that fall in each cell is your hypotheses (as seen in the first table), than they are: 0.35, 0.15 for group1 and 0.3, 0.2 for group 2 with (2-1)*(2-1)=1 degrees of freedom. The effect size you chose by these proportions is quite small (in a sense it is the magnitude in the difference between the observed and the expected values):

library(pwr)

> prob <- matrix(c(.35,.15,.3,.2), byrow = TRUE, nrow = 2)
> ES.w2(prob)
[1] 0.1048285

The significance level you want is 0.05 (I guess). The last piece is the power, which is the 1-P where P is the chance for a type II error (failing to reject a false null hypotheses). This one is tricky, but a good rule of thumb is setting power to 0.9. Given all these:

> pwr.chisq.test(w=0.1048285,df=1,sig.level=0.05,power=0.90)

 Chi squared power calculation 

          w = 0.1048285
          N = 956.1749
         df = 1
  sig.level = 0.05
      power = 0.9

NOTE: N is the number of observations

You need 957 observations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.