It's well known that Pearson correlation can underestimate the true value in case of non-linear relationship. But can it overestimate? For example, Pearson correlation of discontinuous distributions, shown here (no. 5), heavily depends on the outliers. This is the only example I found where linear (Pearson) correlation overestimates the intuitive relation.
But if there are no outliers, can Pearson correlation overestimate the true value? The true value is given by an oracle that outputs (nonlinear) correlation for any two complex distributions (or their infinite realizations X and Y). One of these oracles can be Shannon mutual information.
Consider two random variables $X$ and $Y$, drawn from Bernoulli distribution with probability $p$ and coupling such that $P(y=1|x=1)=1-p$:
from sklearn.metrics import normalized_mutual_info_score def coupled_bern(p=0.1, n=100000): x = np.random.binomial(n=1, p=p, size=n) py = np.where(x, 1-p, p) y = np.random.binomial(n=1, p=py, size=n) return x, y x, y = coupled_bern() nonlinear_corr = normalized_mutual_info_score(x, y, average_method='arithmetic') # 0.37 corr = np.corrcoef(x,y)[0,1] # 0.62
In this example, for any $p$, the linear correlation is always higher than nonlinear (normalized MI). However, I'm not sure that we can say that "linear correlation overestimates the correct value" because we don't know the "correct" value, which is something undefined here.
The question I posed in the title comes from the paper, where the authors claim "when the relation between the two signals is nonlinear, magnitude-squared coherence (MSC) can produce totally misleading results". MSC can be viewed as a linear correlation estimator in the frequency domain. And the plots they have for MSC demonstrate overestimated coherence (read correlation), compared to mutual information.