I've been using an Excel add-on to calculate Spearman's rank correlation coefficients between two variables (bone density and bone survivorship / preservation) with and without a potentially problematic data point. My outputs have been as follows:

  • Bone Density 1: $\rho$ -0.050 p-value 0.784
  • Bone Density 1 with problem data: $\rho$ 0.020 p-value 0.9117
  • Bone Density 2: $\rho$ -0.039 p-value 0.8314
  • Bone Density 2 with problem data: $\rho$ 0.007 p-value 0.9705

If I understand correctly, all of the $\rho$ values indicate very weak correlations. However, the very high p-values indicate that I cannot reject the null hypothesis.

  • What does this mean for the interpretation of the $\rho$ values?
  • Should I say that the rho values indicate weak correlations, but the p-values indicate that I cannot reject the null hypothesis?
  • 8
    $\begingroup$ There is no but here: the correlations are around zero; the P-values are $\gg$ conventional thresholds such as 0.05; therefore you cannot reject the null hypothesis that they are zero. I would suggest that even the term "weak" is wishful thinking: the highest correlation absolutely is 0.05, which is negligible. Also, the problem data don't make much of a difference. What you should always do if you are not doing it already is plot the data so that you can judge whether the correlation is consistent with what you see. $\endgroup$ – Nick Cox Sep 4 '17 at 16:08
  • 1
    $\begingroup$ What you should report depends on how you're expected to report. If a relationship with bone density is what you're expecting, then that hypothesis has a problem. For further guidance, tell us where this lies on the spectrum from high school report, university term paper, etc., to intending to submit this work to a leading journal. $\endgroup$ – Nick Cox Sep 4 '17 at 16:10
  • 1
    $\begingroup$ This is archaeological data, which is part of my dissertation. I'm looking at how well preserved certain skeletal elements are based on the density of the bones. The commonly held hypothesis is that the denser the bone, the more likely it is to survive erosional processes that slowly destroy stuff. We run these tests to see if the bones in the assemblage reflect a pattern where only the densest survive, which indicates we are dealing with a skewed assemblage. If there's no indication that bone density determines preservation, then it means the assemblage is less affected (biased) by erosion. $\endgroup$ – J E Sep 4 '17 at 16:22
  • 1
    $\begingroup$ OK, so a weak relationship is (unusually) good news. Watch out that "skewed" in statistics refers to asymmetric distributions, not problems with sample selection. $\endgroup$ – Nick Cox Sep 4 '17 at 16:27
  • 1
    $\begingroup$ how did you rank bonesurvivor-ship? $\endgroup$ – Martijn Weterings Jul 14 '18 at 12:16

You are misinterpreting the p-value. It actually represents the probability to observe a certain effect or stronger (in your case a correlation) if the null hypothesis - i.e. no correlation - is the correct one. So your results are perfectly consistent: correlation coefficient is close to zero, and p-value confirms that the little correlation apparently visible is actually a statistical fluke. In fact, adding or removing data will change a bit your correlation coefficient, but the p-value tells you that in both instances the correlation is not actually there.

  • 5
    $\begingroup$ Your second sentence in turn is wrong. The P-value is not the probability that the null hypothesis is true; it's the probability of results as or more unfavourable to the null hypothesis as those observed, calculated as if the null is true. I don't think we can do more than guess at the effect of changing the data beyond the fact that the OP has confirmed that changing one bit had little effect. $\endgroup$ – Nick Cox Sep 4 '17 at 17:12
  • $\begingroup$ I agree with Nick Cox -- the statement in your second sentence is not a correct interpretation of what a p-value means (and the third could also use a tweak). However this does give important information in the first sentence. $\endgroup$ – Glen_b Sep 5 '17 at 2:33

These are the values you would expect to see if your data was noise devoid of signal. The near-zero rho means there is no trend present, and the high value of p means that even this lack of a trend is unreliable. It's possible (although unlikely) that there is some structure to this data which is being obfuscated somehow, but the only way we could determine that is if you provided us with a plot.

  • 3
    $\begingroup$ Not quite, as there are many non-random patterns that give rise to negligible correlations. It's my guess that that's unlikely in this case, but we see no plots to have an opinion. $\endgroup$ – Nick Cox Sep 6 '17 at 11:25
  • 2
    $\begingroup$ Yes, I realize it doesn't follow directly that the data must be noise, but given these statistics, the lack of plots, and the general context of the question (biomed data disappointingly often looks like noise), noise seems by far the most likely. I've edited my answer to request a plot. $\endgroup$ – jon_simon Sep 6 '17 at 14:47
  • $\begingroup$ We agree, but absence of evidence is not evidence of absence. $\endgroup$ – Nick Cox Sep 6 '17 at 14:57
  • $\begingroup$ The plots all look like they lack any pattern. I am trying to figure out how to upload them. $\endgroup$ – J E Sep 7 '17 at 1:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.