I once heard the following statements about the "correlation" in the context of regression.
correlation only measures linear relationships
$\text{corr}(X,Y)=0$ does not mean the variables are not related!
My question is why can't correlation measure non-linear relationships?
Secondly, what does $\text{corr}(X,Y)=0$ tell us?
As an opening comment, the word "correlation" sometimes is used in order to indicate the existence of an unspecified form of stochastic dependence -some authors will write "the variables are correlated" and they mean "the variables are dependent". So when one reads such a general statement, one must be careful to interpret it in the context that it is used.
To the other extreme, "correlation" sometimes is used as a shortcut for Pearson's product moment correlation coefficient which is the most widely used related measure, and for which the notation $\operatorname{Corr}(X,Y)$ is employed. But other measures do exist, so again, one should be alert.
The phrase "correlation only measures linear relationships" can be misleading, if one thinks hard enough about it. If it is zero, then we usually say "the relation between the two variables, if it exists is not linear". But if it is not zero, we cannot say that "their relation is linear". It can very well be non-linear and the correlation to be non-zero.
AN EXAMPLE
Consider a random variable $X$ that is continuous uniform $U(0,1)$ and define another random variable by $Y = X^3$. This is a clear non-linear relationship. Their covariance will be
$$\operatorname{Cov}(X,Y) = E(XY) - E(X)E(Y) = E(X^4) - E(X)E(X^3)$$
For a $U(0,1)$ random variable we have
$$E(X) = \frac 12,\;\; E(X^3) = \int_{0}^{1}x^3dx = \frac 14,\;\;E(X^4) = \int_{0}^{1}x^4dx = \frac 15$$
So $$\operatorname{Cov}(X,Y) = \frac 15 - \frac 12\cdot \frac 14 = \frac 3{40} \qquad [1]$$
The Pearson's correlation coefficient is
$$\operatorname{Corr}(X,Y) = \frac {\operatorname{Cov}(X,Y)}{\sqrt {\operatorname{Var}(X)\operatorname{Var}(Y)}}$$
We have
$$\operatorname{Var}(X) = \frac 1{12},\;\; \operatorname{Var}(Y)= E(X^6) - \left(E(X^3)\right)^2 = \frac 17 - \frac 1{16} = \frac 9{112}$$
Bringing it all together we get
$$\operatorname{Corr}(X,Y) = \frac {\frac 3{40}}{\sqrt {\frac 1{12}\cdot \frac 9{112}}} \approx 0.9165$$
... a very high value. Can we adequately express $Y$ as a linear function of $X$? Let's have a look: Based on a random sample of $1.000$ observations from a $U(0,1)$ we get
(note that the theoretical $\beta$ coefficient is $0.9$).
Is the blue straight line an "adequate representation" of the relation between $Y$ and $X$?
I would tentatively say that, if anything, the absolute value of the correlation coefficient measures the "uniformity" of the direction of covariance, rather than its "linear" nature. In our example, when $X$ tends to rise $Y$ rises too, in all cases (this is the nature of the non-linear relationship between them, given also the support of $X$).
Correlation is based on the covariance between two variables (it's basically scaled covariance). Covariance describes the linear relationship between two variables, as such correlation can only describe the linear relationship between two variables. This image should show you how a Cor(X,Y) = 0 can arise in multiple cases where there are no relationship and non-linear relationships between X and Y. Some of the examples are kind of silly, but the point about sinusoidal waves kind of drives it home.
Image by Denis Boigelot, given to the public domain. Source: Wikimedia Commons, https://commons.wikimedia.org/wiki/File:Correlation_examples2.svg
$corr(X,Y)$ is just a scaled covariance so it evaluates, by construction, the linear co-dependence between the variables X and Y. To understand a little bit better how it works take a look at the covariance matrix formula. You'll realize it is a generalization of the variance formula, as it evaluates if two variables value vary in concordance or not. If they do, and their values increase together, the covariance will be positive, if when one decreases the onther decreases in the same way. If the way they vary is unrelated, then the average value calculated in the formula will be zero and we say the variables are not correlated.
You can say this analysis is simplistic, as it does not provide an evaluation of the correlation when different $Y(X)$ or $X(Y)$ are not assumed. Well, the problem is exactly that: to strictly evaluate another dependence you have to assume it, and if you prefer not to do so the analysis of correlation through the covariance is the most natural.
This said, you can evaluate the correlation using different indicators. Take a look at Spearman's Rank, in which you calculate the Pearson's correlation coefficient of the ranked variables, and and a consequence you do not assume a linear dependence but merely a monotonic one. Or even the more recent (and arguably more powerful) distance correlation. The wiki plots similar to the ones showed before will show you the different capabilities of these indicators.
