Given two random variables $\xi$ and $\eta$ we can compute their "correlation coefficient" $c$, and form the line of best fit between these two random variables. My question is why?
1) There are random variables, $\xi$ and $\eta$ which are dependent in the worst possible way, i.e. $\xi = f(\eta)$ and despite this $c=0$. If one only thinks along linear regression, then one would be totally blinded to this.
2) Why linear specifically? There are other kinds of relationships that can exist between random variables. Why single that one out of all others?