I read some other topics on this but the concept still isn't clear to me. Given random variables $X$ and $Y$, I believe the correlation coefficient $\rho=\frac{Cov(X,Y)}{\sigma_X\sigma_Y}$ has an absolute value of 1 if and only if $Y$ is a linear function of $X$. I'm trying to get an intuitive understanding on why $\rho$ measures the strength of a specifically linear relationship and not something else? Is there a way to see this without involving parameters of least squares linear regression?
-
$\begingroup$ Does en.wikipedia.org/wiki/Anscombe%27s_quartet help? $\endgroup$– rolando2Apr 4, 2020 at 10:19
1 Answer
First You can see it by Using the Cauchy-Schwarz inequality. But I use another method.
Define
$$h(t)=E\bigg((X-\mu_x)t+(Y-\mu_y)\bigg)^2\geq 0$$
so
$$h(t)=t^2 Var(X)+2t cov(X,Y)+Var(Y)=at^2+bt+c$$
since $h(t)\geq 0 $ so $\Delta\leq 0$ in hence
$$\big(2 cov(X,Y)\big)^2-4Var(X) Var(Y)\leq 0$$ so $$cov^2(X,Y)\leq \sigma^2_x \sigma^2_y$$
Now
$$|\rho|=1$$
$$\Leftrightarrow$$ $$cov^2(X,Y)= \sigma^2_x \sigma^2_y$$ $$\Leftrightarrow$$ $$\Delta=0$$
$$\Leftrightarrow$$ $$h(t_1)=E\bigg((X-\mu_x)t_1+(Y-\mu_y)\bigg)^2=0$$ where $t_1=\frac{-b}{2a}$
$$\Leftrightarrow$$ $$P((X-\mu_x)t_1+(Y-\mu_y)=0)=1$$
$$\Leftrightarrow$$ almost surely $$Y=-t_1X+(\mu_y+t_1\mu_x)$$
-
1$\begingroup$ I've taken cou to be a typo for cov. Expressions such as cov and Var are more readable if roman, not italic, in my view. $\endgroup$– Nick CoxApr 4, 2020 at 13:02
-