I am comparing scores of two student groups using t-test (in scikit-learn). Each group has different number of students. Since there are considerable numbers of 0 scores in my data, the standard deviation gets even higher than the mean for the groups. So, I wonder, in this case, if it is still acceptable to keep the 0 scores? I appreciate any information!
$\begingroup$
$\endgroup$
4
-
2$\begingroup$ What do the scores mean? Do students actually get scores of 0? The issue for the t test is what the two distributions look like. It doesn't make sense to drop 0 scores if they are real. The t test can be used with unequal sample sizes. It is usually assumed that the two variances are equal when applying the t test for comparing two means. But even in the cases where the two variances are obviously different the Welch test which approximates a t distribution under the null hypothesis can be applied. If the distributions differ greatly from the normal Wilcoxon's rank sum test can be used. $\endgroup$– Michael R. ChernickCommented Jan 11, 2017 at 18:50
-
$\begingroup$ Thanks for the detailed reply. I wish you can post this as answer! So, if scores of 0 are meaningful in my data, then I should keep them, and high variance does not violate anything as far as I understand. $\endgroup$– renakreCommented Jan 11, 2017 at 18:55
-
$\begingroup$ @renake I have put the answer part of my comment into a answer as you requested. $\endgroup$– Michael R. ChernickCommented Jan 11, 2017 at 20:52
-
$\begingroup$ "Having a lot of 0 scores" could be a serious problem if you want the t-statistic to have a t-distribution (i.e. if you want p-values calculated as if you had normally distributed populations to tell you anything) $\endgroup$– Glen_bCommented Jan 12, 2017 at 0:30
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
12
1. If the data come from normal distributions with common variance the t distribution is exactly the correct distribution under the null hypothesis. The sample sizes do not need to be equal (In practice we don't know whether or not these assumptions are true but it may be reasonable assumptions based on other knowledge. For the test statistic use the pooled estimate of variance.
2. Same assumptions as in 1) except the variances are known to be very different, then you use separate estimates of variance and apply Welch's test (approximately t with possibly a non-integer number for degrees of freedom.
3. The fact that there are many zeros means that the normality assumption is suspect. It is not justifiable to remove the zeros because they are legitimate scores. The Wilcoxon rank sum test only requires that the samples are independent and have the same distribution under the null hypothesis. As Bill Huber points out because the test uses ranks and you have zeros that could lead to several ties it may not be the best alternative. Another alternative would be the bootstrap which does not involve ranks and does not require any normality assumption.
answered Jan 11, 2017 at 20:48
-
1$\begingroup$ Isn't the issue whether the sampling distributions of the means are normal, not whether the underlying distributions of the data are normal? One difficulty with any rank sum test in this situation is the potential for a large number of ties at the rank of zero. $\endgroup$– whuber ♦Commented Jan 11, 2017 at 22:21
-
$\begingroup$ @whuber I was trying to give the OP an understanding of some of the assumptions that might lead to a t test being acceptable. I gave most of this in my comment and the OP encouraged me to put it in an answer. The other issue which seemed to be more important to the OP was whether or not the zeros should be kept. With regard to Wilcoxon I agree that ties coming about because of the zeros can be a problem. I could easily also have recommended bootstrap. Mainly I wanted the OP to understand that there are alternatives. $\endgroup$ Commented Jan 11, 2017 at 22:48
-
$\begingroup$ Regrading the sampling distribution being normal, that will occur if the data are normal. My point was that if the data is approximately normal the t test will work. I wasn't going to go into the CLT. It depends on the sample size and how far from normal the origimal samples are. $\endgroup$ Commented Jan 11, 2017 at 22:51
-
1$\begingroup$ Those issues would seem to be immediately relevant and of interest to the OP. Unless they are addressed, it seems difficult to justify a blanket recommendation of the Wilcoxon RS test. Furthermore, if your interest lies in helping the OP understand the test, wouldn't it be better to focus the discussion on what really matters--the sampling distribution--rather than something that is less directly relevant (the data distribution)? $\endgroup$– whuber ♦Commented Jan 11, 2017 at 22:56
-
1$\begingroup$ @whuber I didn't realize that I made such a strong recommendation. I will edit the post! $\endgroup$ Commented Jan 11, 2017 at 22:58