1
$\begingroup$

I have data that includes the number of students in a class and the percentage of that group who achieved a preset pass level in a standard test. I have this data for a number if different schools in two population samples, about 30 schools in each. The class sizes differ considerably, so it seems take sense to use the percentage already given when calculating the t-test.

But I also know that percentages shouldn't be averaged. I could calculate the number of students from the data given, but this does not reflect class size, which seems important. The percentage "automatically" reflects the weighting of class size. Any advice or thoughts about this problem appreciated.

Example data to illustrate problem

No students    percent passed     calculated no passed

28                7%                2

79                7%                6

28               51%               14

58               50%               29

Thanks Tim

$\endgroup$
  • 3
    $\begingroup$ What are you trying to learn about, Tim? You mention "two population samples." This suggests you might not be interested in deciding whether two particular schools differ (obviously there are some huge differences even in your small example) but perhaps you would like to know whether the two populations have significantly different distributions of passing proportions? $\endgroup$ – whuber Jul 23 '12 at 16:48
  • 2
    $\begingroup$ Like @whuber, I think we need more information. But as a heads-up, I suspect you may ultimately need to use a multilevel generalized linear model. If you are not familiar with these, you could get an overview from some answers I have recently given, here (for a little about generalized linear models), & here (for a little about the multilevel options for GLiMs). However, if you are unfamiliar w/ these, you will probably need to work with a consultant. $\endgroup$ – gung - Reinstate Monica Jul 23 '12 at 17:05
  • $\begingroup$ I want to check whether the sample mean of to groups is drawn from the same population, using a t-test as the significance test. Not sure if this is how to do it? Tim $\endgroup$ – Tim Rowan Jul 23 '12 at 18:52
  • 1
    $\begingroup$ @TimRowan, in your comment and question you refer to "mean", what is this the mean of? Is it the mean pass rate of individual, or schools? or something else (eg average score on the original test). $\endgroup$ – Peter Ellis Jul 23 '12 at 20:16
  • $\begingroup$ Okay but the OP sounded like he wanted to do a simple t test comparing proportions. But my answer is defective because as Peter points out this is more complicated than would be accomplished by the tests I am suggesting. I think I should have looked at parts of the question more carefully and I am going to graciously drop my answer and apologize to my friends Gung and Huber for my hasty comments. The only way I could see a t test being remotely possible would be the scenario I imagined. Withdrawn. Those downvotes were well administered! See I can recognize my flaws. $\endgroup$ – Michael R. Chernick Jul 24 '12 at 0:07
8
$\begingroup$

There are several issues here.

Q1. Can a t-test be used with data that is a percentage?

A - Strictly speaking a t-test should be used when the underlying population has a normal distribution and you have to estimate the mean and variance simultaneously. Your percentage pass rates are not going to normally distributed, but they may well be close enough that no harm is done by using a t-test. This is unlikely to be the big obstacle.

Q2. What is the right way to aggregate the percentage pass rates into an overall pass rate?

A - you should use a weighted average of the percentages, with the weights being the class sizes. This will give you a percentage (for each of your two groups of classes) that is equivalent to their overall individual percentage pass rate. There are several equivalent ways of calculating this; one is by a weighted average, the other is (as you have started doing) converting each class score into a number of passing students and a total number of students, summing these numbers and calculating the final percentage.

There are complications in estimating the variance to use for a t-test but I will ignore these because of Q3 below:

Q3. (implied) - is this procedure as a method of comparing pass rates between two groups appropriate?

A - No. The problem is that there are known to be very significant class (teacher) and school effects in educational performance. No simple t-test can take these into account, and a t-test will give you a big chance of showing something as significant when it isn't. Basically, the data in these grouped situations is not worth as much as its sample size suggests. This is because 30 students in one class are not really 30 independent data points - they may all be reflecting skill of the teacher (even putting aside other important known environmental variables, and their individual socio economic background). The accepted way to do the sort of inference you want is with a multi-level model, a special case of a mixed-effects model, which unfortunately is much harder to implement than a t test, but will give you much more reliable results.

$\endgroup$
  • $\begingroup$ Thank you very much. This answers all of my questions. Tim. $\endgroup$ – Tim Rowan Jul 24 '12 at 21:07
  • 1
    $\begingroup$ Hi Tim, it appears that you are pleased with this answer - please consider accepting/upvoting it :) $\endgroup$ – Macro Jul 25 '12 at 12:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.