Please explain what is the difference between if two variables are linearly dependent or linearly correlated.
I looked up the wikipedia article but didn't get a proper example. Please explain it with example.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ what does linear mean here? In pure maths usually it means something like this $T(x+y) = T(x) + T(y)$ or at least intuitively a line/matrix e.g. $y = ax $. Is it what it means here and if yes in what formal sense is it "linear"? $\endgroup$– Charlie ParkerCommented Oct 8, 2021 at 19:17
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
4
Two variables are linearly dependent if one can be written as a linear function of the other. If two variable are linearly dependent the correlation between them is 1 or -1. Linearly correlated just means that two variables have a non-zero correlation but not necessarily having an exact linear relationship. Correlation is sometimes called linear correlation because the Pearson product moment correlation coefficient is a measure of the strength of the linearity in the relationship between the variables.
answered Jun 28, 2012 at 9:38
-
4$\begingroup$ +1. Though, I'd rather say the Pearson coef. "is a measure of strength of linear relationship" instead of
is a measure of the degree of linearity in [= of?] the relationship$\endgroup$– ttnphnsCommented Jun 28, 2012 at 9:59 -
$\begingroup$ @ttnphns Okay that sounds more appropriate. $\endgroup$ Commented Jun 28, 2012 at 10:51
-
$\begingroup$ Perhaps $\rho^2$ rather than $\rho$ would be a better measure since we don't need to hassle with $\rho$ close to $-1$ meaning a strong linear relationship (albeit with negative slope). Also, consider how much variance is explained versus non-explained, and that $\rho = 0.51$ does not provoke the statistician into turning cartwheels and doing handstands in celebration whereas $\rho^2 > 1/\sqrt{2} \approx 70\%$ is much better evidence of a positive (read, publishable) result. $\endgroup$ Commented Jun 28, 2012 at 20:22
-
$\begingroup$ what does linear mean here? In pure maths usually it means something like this $T(x+y) = T(x) + T(y)$ or at least intuitively a line/matrix e.g. $y = ax $. Is it what it means here and if yes in what formal sense is it "linear"? $\endgroup$ Commented Oct 8, 2021 at 19:17
$\begingroup$
$\endgroup$
1
In $\mathbf{R}^2$ linear dependence implies that one vector is a linear function of the other: $$ \textbf{v}_{1}=a\textbf{v}_2. $$ It's clear from this definition that the two variables would move in lock-step, implying a correlation of $1$ or $-1$ depending on the value of $a$. To more fully understand the differences and connections between the concepts, however, I think it's beneficial to consider the geometry involved.
The graph below shows an example of the formula for linear dependence. You can see that the vectors are linearly dependent because one is simply a multiple of the other.
This is in contrast to linear independence, which in $\mathbf{R}^2$ is described by:
$$
\textbf{v}_{1}\neq a\textbf{v}_2
$$
for vectors $\textbf{v}_1, \textbf{v}_2 \neq \textbf{0}.$ An example of linear independence can be seen in the graphic below.
The most extreme version of linear independence is orthogonality, defined for vectors $\textbf{v}_1, \textbf{v}_2$ as: $$ \textbf{v}_{1}^T \textbf{v}_{2} = 0. $$ When graphed in $\mathbf{R}^2$, orthogonality corresponds to the vectors $\textbf{v}_{1}$ and $\textbf{v}_2$ being perpendicular to one another:
Now, consider Pearson's correlation coefficient: $$ \rho_{\textbf{v}_{1}\textbf{v}_{2}} = \frac{(\textbf{v}_{1}-\bar{v}_{1}\textbf{1})^T(\textbf{v}_{2}-\bar{v}_{2}\textbf{1})}{\sigma_{\textbf{v}_{1}}\sigma_{\textbf{v}_{2}}}. $$
Note that if the vectors $(\textbf{v}_{1}-\bar{v}_{1}\textbf{1})$ and $(\textbf{v}_{2}-\bar{v}_{2}\textbf{1})$ are orthogonal then the numerator of Pearson's coefficient is zero, implying that the variables $\textbf{v}_{1}$ and $\textbf{v}_{2}$ are uncorrelated. This illustrates an interesting connection between linear independence and correlation: linear dependence between the centered versions of the variables $\textbf{v}_{1}$ and $\textbf{v}_{2}$ corresponds to a correlation of $1$ or $-1$, non-orthogonal linear independence between the centered versions of $\textbf{v}_{1}$ and $\textbf{v}_{2}$ corresponds to a correlation between $0$ and $1$ in absolute value, and orthogonality between the centered versions of $\textbf{v}_{1}$ and $\textbf{v}_{2}$ corresponds to a correlation of $0$.
Thus, if two vectors are linearly dependent the centered versions of the vectors will also be linearly dependent, i.e. the vectors are perfectly correlated. When two linearly independent vectors (orthogonal or not) are centered the angle between the vectors may or may not change. Thus for linearly independent vectors the correlation may be positive, negative, or zero.
-
$\begingroup$ what does linear mean here? In pure maths usually it means something like this $T(x+y) = T(x) + T(y)$ or at least intuitively a line/matrix e.g. $y = ax $. Is it what it means here and if yes in what formal sense is it "linear"? $\endgroup$ Commented Oct 8, 2021 at 19:17
$\begingroup$
$\endgroup$
2
Let f(x) and g(x) be functions.
For f(x) and g(x) to be linearly independent we must have
a*f(x) + b*g(x) = 0 if and only if a=b=0.
In other words there is no c such that a or b is not zero but
a*f(c) + b*g(c) = 0
If there is such a c, then we say that f(x) and g(x) are linearly dependent.
e.g.
f(x) = sin(x) and g(x) = cos(x) are linearly independent
f(x) = sin(x) and g(x) = sin(2x) are not linearly dependent (Why?)
-
2$\begingroup$ With the definition you're using there, there can be a $c$ such that $a f(c) + b g(c) = 0$; they're only linearly dependent if it happens for all $x$ in the considered domain; for example, consider your second example, with $c=\pi/3$. (Also, I think there's a problem with your first example) $\endgroup$– Glen_bCommented Nov 15, 2013 at 0:53
-
$\begingroup$ what does linear mean here? In pure maths usually it means something like this $T(x+y) = T(x) + T(y)$ or at least intuitively a line/matrix e.g. $y = ax $. Is it what it means here and if yes in what formal sense is it "linear"? $\endgroup$ Commented Oct 8, 2021 at 19:17