1
$\begingroup$

I have data for a school class of grades and family income(*). To see whether the richest kids do better, I find the spearman rank correlation coefficient for the grades and income of the kids in the class. To figure out if the result is statistically significant I can consult a table.

Now suppose I have this data for K classes of different sizes (ranging from 10 kids to 40). K might be in the hundreds or thousands. I have the spearman coefficient for each class. How shall I proceed?

  1. I take the mean of these coefficients - is there a way to decide if it's significant? Say the mean is 0.03, too low to be significant for any particular class - but averaged over so many might it be significant?

  2. Is it much more sensible to lump the data together and find just one spearman coefficient for all the data?

  3. What if I can't lump the together (e.g. they're graded on different scales so it only makes sense to compare kids in the same class as each other)? How can I use all this data to check whether there's a statistically significant relationship between household income and grades?

(*) these details are fake but I'm facing an equivalent problem.

$\endgroup$
2
  • $\begingroup$ What hypothesis are you trying to test? Income causes grades to be higher? If you are interested in the association between grades and income, you should use a linear model predicting grades from income. Calculating correlation for each grade is inferior to regression with adjustment or stratification for grade level. $\endgroup$
    – Todd D
    Commented Jun 15, 2017 at 17:49
  • $\begingroup$ @Todd: Computing the correlation between grade and income is equivalent to running a linear model. He has chosen to correlate on ranks rather than values, which is equivalent to running a linear model on the ranks. $\endgroup$ Commented Jun 15, 2017 at 18:05

2 Answers 2

1
$\begingroup$

You don't want to take the mean of the coefficients. Each coefficient has a probabilistic interpretation only for a specific n. Your n's are all different, and even if you could combine them in some not-easy-to-compute way, the "effective n" of the aggregate would be higher, in some not-easy-to-compute way, than the individual n's.

You want to form one big sample and use it to compute one correlation coefficient. Your main worry here is that grades don't mean the same thing for different classes. This is an entirely reasonable worry to have. To get around it, you'll need to change the question you ask. For example, instead of considering the grade itself as the variable, you might consider the variable to be the deviation of the grade from the mean grade given by that teacher.

By the way, the classic Spearman test assumes no ties. There are some ad hoc approaches to dealing with ties, but I would not be too confident in them if there are lots of ties (e.g. 1000s of students and only 5 possible grades). You could use bootstrapping to get around this. Also, taking my suggested deviation-from-mean-grade approach will lessen this problem.

$\endgroup$
2
  • $\begingroup$ can you comment on how a given correlation coefficient has a different interpretation based on the sample that generated it? It would seem that rho=0.75 would have the interpretation regardless of n. $\endgroup$
    – Todd D
    Commented Jun 18, 2017 at 21:42
  • $\begingroup$ @Todd: Certainly $\rho$ has an interpretation as a correlation coefficient independent of $n$. But it only has a probability interpretation, as the chance of observing such a large value under the null hypothesis of no correlation, for a specific $n$. Another way to think of this is that for your measurement to be useful, you need to not only report not on a value for $\rho$, but also an error bar, and how to compute that error bar by combining different $\rho$ with different $n$ is not clear. $\endgroup$ Commented Jun 19, 2017 at 18:46
0
$\begingroup$

What matters for statistical inference is the distribution of the variable and characteristics such as independence. The variable that is measured (height, weight, class size, Spearman Rho) is irrelevant. Unless your distribution is extremely non-normal, a single sample t test would be OK.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.