If relationships between two or more variables are found by Support Vector Machines, Random Forests, Decision Trees, and/or Extreme Learning Machines, could a simple Pearson's Correlation also detect correlations between those variables?
If yes, why? If not, why not?
I'd be thankful if you had a source I could quote for my thesis.
NO
Consider points forming a symmetric parabola in the $x$-$y$ plane. Many machine learning models, including a linear regression using a quadratic term, will catch this relationship. However, the Pearson correlation will be (correctly) reported as zero.
A reference could be example 4.5.9 in the second edition of Statistical Inference by George Casella and Roger L. Berger, page 174.
Further, once you have multiple variables like you probably do in a machine learning problem, it is not clear what a Pearson correlation between all of them would be, since classical Pearson correlation inputs two variables, not $3+$.
Using Pearson Correlation is same using Univariate Linear Regression, they both rely on the x variable and y variable satisfy linear connection. However, this is not always the case in real problems, sometimes you need multi linear regression to include many variables or you need advanced machine learning models to detect complex connections.
