1
$\begingroup$

I have data where there are 2 groups, 100 measurements per group and each measurement is a proportion (or number out of 1000) and I want to test for a statistically significant difference between the two groups. Can I use a t-test on the means, or does the fact that the values are proportions cause problems (meaning I would need a z-test)? But is there a problem with doing a z-test on means of proportions rather than raw populations?

Is it correct to say that because each measurement is a sample proportion and therefore each group represents a sampling distribution of the sample proportion, the mean proportion from all the measurements should be a good estimate of the population proportion? If this is true, surely I can use the mean proportion and standard deviation of each group for my z-test, instead of calculating the standard error to estimate the standard deviation, since I have way more than just one measurement of the proportion?

Just a note as well, I am not comfortable with treating all my samples as one large sample per group, as the proportion actually represents the results of a classification algorithm, in which each trial had some level of inbuilt randomness and the proportion of correct classifications is on the same data set for each trial.

Edit: To be clear, here is roughly what my data looks like:

enter image description here

As stated, each number is a number of 'success' trials out of 1000 and there are 100 of these measurements per group (with a difference in the independent variable).

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ If your denominators are all equal, your data points are just in a unit 1000 times larger than originally (more or less), something like kilometers instead of meters. If you want to compare 60/1000, 37/1000, etc, just compare 60, 37, etc. Since the numerators are counts, you may want to do your test with the assumption of a Poisson distribution instead of the normal distribution that’s assumed for a t-test. $\endgroup$
    – Dave
    Apr 18, 2020 at 0:52
  • $\begingroup$ @Dave Do you mean comparing the mean counts of each group using a poisson distribution? $\endgroup$
    – ajax2112
    Apr 18, 2020 at 0:56
  • $\begingroup$ What I really mean is a Poisson regression with the group variable as the predictor, at least as a start. (You may end up preferring negative binomial, for instance.) You’re the one who know how your data look, though, and if a normal approximation is reasonable. $\endgroup$
    – Dave
    Apr 18, 2020 at 1:33

1 Answer 1

Reset to default
1
$\begingroup$

DO you mean proportion with 100 as the denominator or 1000?

Either way, with two groups, you can use a z-test for the difference between proportions which would be equivalent to using a chi-squared statistic with 1 df for independence of two binary variables to compare p1=p2.

This is because p1=p2 rearranges to p1-p2=0 and because the z-statistic squared would be asymptotically equivalent to the chi-squared test of independence for 2 binary variables (resulting in 1 proportion per group).

The "mean" of the 1/0 responses would equal the proportion of "1" responses, but generally you would frame your test in the comparison of proportions.

It isn't really clear what you mean that "each group is a sampling distribution"-- each sample statistic, in this case a proportion, has a sampling distribution. I would, for all purposes, split in your mind "mean" for quantitative variables and "proportion" for categorical variables.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I mean a proportion where 1000 is the denominator, because each measurement of the 100 measurements per group is effectively a sample of 1000. When I say each group is a sampling distribution, what I mean is each measurement is a sample proportion, so I have the entire group as the distribution of the sample proportion. The mean for each group is therefore the mean of the sampling distribution. $\endgroup$
    – ajax2112
    Apr 18, 2020 at 0:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.