what exactly is a correlation of zero meant to be, does that mean that changes in x does not affect y, if that is the case, from the scatter plot
That is completely false, as sometimes as x increases, y decreases or increases
$\begingroup$
$\endgroup$
3
asked Jun 17, 2021 at 18:22
-
$\begingroup$ Correlation is the measure of how random variables are related to one another. There are some coefficients trying to measure the degree of correlation. If you are talking about Pearson correlation coefficient it gives information about linear relationships, so if it is zero it means that there is no linear relationship between $x$ and $y$. For nonlinear relationship Pearson correlation coefficient is noninformative, as an example consider this relation $x^2+y^2=1$, here correlation coefficient is zero. $\endgroup$– D.GJun 17, 2021 at 20:06
-
1$\begingroup$ $X \perp Y \implies \text{R}[X,Y] = 0$ but the converse is does not hold. $\endgroup$– GalenJun 17, 2021 at 20:15
-
1$\begingroup$ The typical example of that reverse implication not holding is a parabola like $y=x^2$. $\endgroup$– DaveJun 17, 2021 at 20:16
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
From JMP: Introduction to Statistics: Correlation:
What is correlation?
Correlation is a statistical measure that expresses the extent to which two variables are linearly related (meaning they change together at a constant rate). It’s a common tool for describing simple relationships without making a statement about cause and effect.
Key points to highlight from that is that a correlation describes the relationship between two variables, assumes the relationship is linear, and it does not make any assumptions about cause and effect.
Thus, firstly addressing what you stated, we can never say whether x does or does not affect y. If we say "affect," we would be implying cause and effect, where x is a cause to effect y. A classic example of a correlation-not-causation is the negative relationship between pirates and global warming; the correlation does not mean the reduction of pirates over the years causes an increase in global temperatures.
(Figure from Wikipedia)
The correlation coefficient describes the strength and direction of the relationship between two variables that are linearly related. When the correlation is zero, it means the two variables are not related, assuming the relationship would have been linear. If you have a scatter plot with a perfect U-shape relationship, for example, where y is seen to sometimes increase and sometimes decrease as x increases, you will get a correlation coefficient equal to zero, but it clearly violates the assumption of linearity. The figure below shows a couple of examples that violate the assumption. Practically, the correlation coefficients wouldn't make sense there.
(Figure from JMP)
The further away the correlation coefficient is from zero (towards -1 or +1), the stronger the relationship between two variables is. Generally speaking, when there is a relationship, the relationship is stronger when the x and y values are closer to the line of best fit (least squares) on a scatter plot (see the figure below). In other words, the correlation coefficient can also be interpreted as the "goodness of fit" of the best fit line.
(Figure from QuestionPro)
A more thorough description about correlations can be found at JMP: Introduction to Statistics: Correlation.
-
1$\begingroup$ This is a really great start! Perhaps discussing the geometric interpretation of the coefficient would also be beneficial. $\endgroup$– GalenJun 17, 2021 at 20:13
-
$\begingroup$ Thanks, @Galen. I added something new to the post. $\endgroup$ Jun 17, 2021 at 21:05
-
$\begingroup$ Nice! I was also thinking about the fact that $\text{Corr}[X,Y] = \cos \theta$ where $\theta$ is the angle between the residual vectors. This speaks to whether the residuals are (anti)parallel, orthogonal, or in between. $\endgroup$– GalenJun 17, 2021 at 21:08
-
1$\begingroup$ Perhaps it is too arcane anyway. If anyone is interested, this article seems to cover what I am referring to. $\endgroup$– GalenJun 17, 2021 at 21:46
-
1$\begingroup$ Briefly stated, you can think of the correlation as an inner product divided by the product of the 2-norms of the residuals. This leads to an angle, which for arbitrary dimensions is defined to be the cosine of the angle between the vectors. $\endgroup$– GalenJun 17, 2021 at 21:48