I am regressing one variable on 1,500 features. I have 30,000 rows of data.
I know some of my features are correlated, but a Ridge regression where I select the ridge parameter $\lambda$ by cross validation (I use Python scikit learn) gives a very small parameter : $1e-13$. So I am doing a vanilla linear regression.
I suspect spurious regression. When I take at random only 20 features for example, my $R^2$ falls near 0, but as soon as the number of features is huge (700 for example) I have a very good $R^2$ on the test data, even if my features are correlated (many of them are moving averages of different periods).
Can I trust my $R^2$, especially that it is calculated on test data and use the model for prediction, or am I doing something wrong?