0
$\begingroup$

I am hoping to understand best way to test statistical significance between 2 dependent population groups.

For example, consider a usability test. When 100 subjects were tested, 50 of them clicked (=50% click rate). However, 50 of the subjects were male, 40 of whom clicked for an 80% click rate for males.

The question is that 80% statistically significant? In other words, do men click more than the population as a whole? I think I need to use a paired $t$-test, however unsure as what I would use as the mean, since these are all population proportions.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Do you mean 50 of the 100 subjects? And significantly different from what? Do you want to know if the part is different from the whole?...the remaining part? $\endgroup$
    – John
    Commented Sep 5, 2013 at 4:17
  • $\begingroup$ "The question is that 80% statistically significant?" against what null? It seems like you want a two sample test, but you must clarify. $\endgroup$
    – Glen_b
    Commented Sep 5, 2013 at 5:14
  • $\begingroup$ The statement I want to make is that males click rate is 700% better then those who are not male. (40/50)-(((50-40)(100-50)))/(40/50)). However if the males subject population was only 1, and that 1 person clicked, i would have a much larger value, but it wouldn't be significant because 1 of 1 is not large enough to be deemed statistically significant. $\endgroup$
    – BigOrangeSU
    Commented Sep 5, 2013 at 5:43
  • $\begingroup$ Do you mean that 40/50-(50-40)*(100-50)/(40/50)=-624.2? The male click rate is 40/50=80%, and the female click rate is (50-40)/(100-50)=20%? If 1 of 1 male clicked, you mean that you will have a larger male click rate (from 80% to 100%), but what becomes insignificant? male click rate = 0 or male click rate = female click rate, or others? Please edit your question to clarify. $\endgroup$
    – Randel
    Commented Sep 5, 2013 at 7:07

1 Answer 1

Reset to default
1
$\begingroup$

You seem to have gone on a convoluted route to asking for how to assess independent count data. You've got 100 independent items. There are 40 males that clicked, 10 males that didn't click, 10 non-males that clicked, and 40 non-males that didn't click. You can easily construct what is called a contingency table (below) from those data and do a $\chi^2$ (chi-square) test for independence. 

         male non-male
click    40   10
no click 10   40

Searching for the chi-square test on the internet will show you the formulas, logic, and even online calculators that can solve the problem.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Sorry I don't have a lot of stats knowledge but from my general understanding, the populations are dependent since the male population is a subset of the total population of subjects. If the populations were independent could I use a binomial approx and check for the result being within 3 std deviations? Sorry I am totally out of my element here, but I am trying to determine statistical significance for a bunch of different usability tests where I want to know if the subset of users proportion is statistically significant to the larger population of subjects. $\endgroup$
    – BigOrangeSU
    Commented Sep 5, 2013 at 17:57
  • $\begingroup$ Yes, the males are a subset of the total population but given that kind of a rule for dependence any values are dependent. Imagine I rolled a die 20 times and got 4 6's. The number 4 is dependent on the fact that I rolled 20 times. But none of those rolls is dependent on any other roll. They're independent rolls. And yes, you could use a binomial approximation and reporting a CI would be a good idea. However, that runs into some judgments on what's conservative and not. I believe it's a better way to report results but not a better test than $\chi^2$. $\endgroup$
    – John
    Commented Sep 5, 2013 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.