I am comparing scores of two student groups using t-test (in scikit-learn). Each group has different number of students. Since there are considerable numbers of 0 scores in my data, the standard deviation gets even higher than the mean for the groups. So, I wonder, in this case, if it is still acceptable to keep the 0 scores? I appreciate any information!
1. If the data come from normal distributions with common variance the t distribution is exactly the correct distribution under the null hypothesis. The sample sizes do not need to be equal (In practice we don't know whether or not these assumptions are true but it may be reasonable assumptions based on other knowledge. For the test statistic use the pooled estimate of variance.
2. Same assumptions as in 1) except the variances are known to be very different, then you use separate estimates of variance and apply Welch's test (approximately t with possibly a non-integer number for degrees of freedom.
3. The fact that there are many zeros means that the normality assumption is suspect. It is not justifiable to remove the zeros because they are legitimate scores. The Wilcoxon rank sum test only requires that the samples are independent and have the same distribution under the null hypothesis. As Bill Huber points out because the test uses ranks and you have zeros that could lead to several ties it may not be the best alternative. Another alternative would be the bootstrap which does not involve ranks and does not require any normality assumption.