0
$\begingroup$

I am trying to learn a bit of R and statistics so as "toy project" I have started from the famous Iris dataset (150 rows, 3 classes). I have applied to two features the "correlation test" in this way:

cor.test(iris$Sepal.Length, iris$Petal.Length)
# 
#   Pearson's product-moment correlation
# 
# data:  iris$Sepal.Length and iris$Petal.Length
# t = 21.646, df = 148, p-value < 2.2e-16
# alternative hypothesis: true correlation is not equal to 0
# 95 percent confidence interval:
#  0.8270368 0.9055083
# sample estimates:
#       cor 
# 0.8717542 

which uses by default the Pearson's correlation.

Based on 2 hypothesis how do I interpret the confidence interval? I tried to use a t-table but I am still not sure if the value "0.8717542" is a good value or not based on the df.

I am using the t-table here (pdf).

I cannot see the DF = 148 but from 100 to 1000 looks there is not too much shift. So is it correct if I affirm that the interval of confidence is between 60% and 70% and based on that, there is an acceptable positive correlation between the two features?

$\endgroup$
5
$\begingroup$

The correlation between Sepal.Length and Petal.Length is $r_{SP} = 0.872$. However, that is a point estimate. It is perhaps more likely than any other point value, but that doesn't mean it's actually the true value of the correlation in the population. Thus, R has also provided a 95% confidence interval around that estimate, $(0.827 \le r_{SP} \le 0.906)$. This is interpreted as any other frequentist confidence interval would be interpreted. Namely, if you were to repeat the data gathering process and analysis identically, over and over, ad infinitum, 95% of the intervals formed the way that one was would be expected to cover the true value of the correlation. With respect to this sample, you can think of it as a measure of the precision of the estimate, in the sense that if your null hypothesis had been any value (not just $0$) that lay outside that interval, your test would still have been significant.

I think part of your confusion pertains to the question of how to test if an $r$-score is significantly different from $0$. One way to do so is to use a $t$-test. (That's what R is doing.) In that context, you don't assess the fitted $r$ value, you convert that $r$-score into a test statistic that is distributed as $t$. The equation is:
$$ t = r\sqrt{\frac{n-2}{1-r^2}} $$ Although you are correlating two variables, this isn't a 2-group $t$-test. It is a one-sample $t$-test. At any rate, R has done that for you. The value of your test statistic is $t = 21.646$. To see how extreme that value is, you can compare it to a central $t$ distribution with $148$ degrees of freedom. When you do so, you will see that the $p$-value is $p < 2.2\times 10^{-16}$.

$\endgroup$
  • $\begingroup$ thank you for your very interesting and detailed answer. How do I interpret the value "p" at this point? Any related link or reading to suggest about all of this? $\endgroup$ – Randomize Apr 28 '16 at 8:06
  • $\begingroup$ @Randomize, the p-value is the probability of getting a result as extreme or more extreme than yours, if the null hypothesis is true. See: What is the meaning of p values and t values in statistical tests? In your case, the probability that $r\ge |0.872|$ or $t\ge|21.6|$. $\endgroup$ – gung - Reinstate Monica Apr 28 '16 at 9:13
1
$\begingroup$

I guess I don't really follow the question, but the "0.8717542" is the Pearson correlation estimate (link). 0.8 is usually considered a very strong correlation as 1 is a perfect correlation.

$\endgroup$
  • $\begingroup$ That is only the correlation, you still need to run a test to check the significance levels and probability. You can get more details here: sthda.com/english/wiki/… $\endgroup$ – RaRdEvA Nov 15 '18 at 2:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.