Consider the example below.
N <- 1000
x1 <- runif(N, -2, 2)
x2 <- runif(N, -2, 2)
x3 <- runif(N, -2, 2)
y <- 0*x1 + 0*x2 + 0*x3 + x3^2
cor(x1, y) # -0.02706921
cor(x2, y) # 0.02323476
cor(x3, y) # -0.001507549
Here, there are three possible variables (
x3) to predict the outcome (
y). The correlation between each predictor variable and
y is close to zero and would be eliminated by your proposed method of screening out variables that have a low correlation with
y. However, squaring
x3 is a perfect predictor of
y. If you have a flexible model that can catch these kinds of nonlinear functions of the original data, you are depriving the model of data that could be highly predictive when used the right way (which a good model should figure out).
For instance, a neural network is able to detect the nonlinear relationship between
y without being explicitly programmed to look for such a relationship, yet screening based on the correlation would deprive such a model of that crucial
L <- nnet::nnet(y ~ x1 + x2 + x3, size = 3, linout = T)
(Yes, there are alternatives to the Pearson correlation used here. Spearman correlation could be an alternative, though the Spearman correlations here are all quite low, too, and would not be particularly helpful.)
Is it ok to employ linear correlation to dismiss some variables because of their LINEAR relationship to then use a model which not necessarily models a linear relationship?
The example above hopefully demonstrates how such an approach can do serious harm to your analysis.