# spurious regression?

I am regressing one variable on 1,500 features. I have 30,000 rows of data.

I know some of my features are correlated, but a Ridge regression where I select the ridge parameter $\lambda$ by cross validation (I use Python scikit learn) gives a very small parameter : $1e-13$. So I am doing a vanilla linear regression.

I suspect spurious regression. When I take at random only 20 features for example, my $R^2$ falls near 0, but as soon as the number of features is huge (700 for example) I have a very good $R^2$ on the test data, even if my features are correlated (many of them are moving averages of different periods).

Can I trust my $R^2$, especially that it is calculated on test data and use the model for prediction, or am I doing something wrong?

• Why do you think something is wrong? Maybe the first half of the features happened to be more informative than the second half, and no single feature is very informative. This doesn't seem crazy. Oct 16 '15 at 7:41
• It is because I know my features are correlated. A PCA of the features reveals that the first 50 eigen values are large (between 1 and 300,000), whereas the others are very very small (near 1e-12). So I know the matrix $X^{T}X$ in the regression equation is ill-conditioned. But still, scikit learn does not find a significant ridge parameter, and my results have goor $R^2$ on test data Oct 16 '15 at 7:55
• Thanks Cagdas. My problem here is that generally a bad designed regression may result in a high $R^2$ for the training data but clearly fails for the test data. Here, I know that my regression is badly designed : correlated features, overlapping periods for mobile averages, but I still have high $R^2$ on every day of data (training on half of the day, test on the other half). Nevertheless, my parameters are not ststable from day to day. Oct 16 '15 at 10:04