I have data for a TV show that runs on multiple days. I know many viewers came from a certain channel. So on day 1 there might for example be 30 viewers coming from channel 1 and 20 viewers coming from channel 2. On day two 10 viewers come from channel 1 and 5 from channel 3 etc. (This is just an illustration, there are a lot more channels). I want to know if the rank of the incoming channels is stable. I therefore paired up the shows on consecutive days and ran a Spearman Rank Correlation. Using scipy I also get the p-values for those correlations.

I want to make a statement over the set of those shows. I do not want to look at one pair, but at all the pairs of shows. I would like to be able to say "We reject the null hypothesis that the rank correlation for consecutive days is 0 with a confidence level of X". So how do I do this for a set of p-values?

One idea I had was that I could make a summary statistic for the p-values. Say for example Y percent of the p-values are smaller than 0.05, or take the average p-value. But I don't know what the correct way to go about this is.


1 Answer 1


This is most easily and meaningfully done using the bootstrap. For each of, say, 1000 ordinary bootstrap resamples (samples with replacement), compute the ranks of all the Spearman correlation coefficients. For each coefficient compute the 0.95 quantiles of the ranks over the 1000 values.

  • $\begingroup$ Thanks for your answer! But I'm afraid, I'm still a bit lost. The way I understand your answer, I still have multiple values but now from a resample (not the original sample anymore). Is there an example you could point me to somewhere? $\endgroup$
    – Basil
    Commented Sep 20, 2015 at 13:22
  • 1
    $\begingroup$ The results would look like this: The observed Spearman rho for variable x17 vs. variable y is 0.3 which placed it at a rank of 700 out of 800 correlation coefficients computed (only 100 were larger). The 2.5th and 97.5th percentiles of the ranks over 1000 bootstrap repetitions of this process were [400, 790]. So speaking loosely we can be 0.95 confident that this pair of variables is not in the 10 top pairs of variables in terms of rho. The data are consistent with the pair not being in the worst 399 pairs. Repeat this process for each pair; look especially at the top pair. $\endgroup$ Commented Sep 20, 2015 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.