Adjusted R2 Validity for Big amounts of observations I am working with a dataset that has a big amount of observations (2000). The purpose of my work is to find which dependent variables (x1, x2, x3...) are linked to my independent variable (y). I have discovered the adjusted R2 method and it seems pretty accurate for what I am looking for since it makes a ponderation of the influence of new variables in the prediction.
The equation for the Adjusted R2 method is shown in 1:
R2_Adj = 1 - [(1-R2)(n-1)/n-k-1]          [1]
where:

*

*N is the number of points in your data sample.

*K is the number of independent regressor.

Apparently it should work. However, when I have a big amount of data: N = 2000 the difference between 1 or 2 regressors is sometimes not big enough to spot a good regressor and instead, sometimes the model accepts a 'bad' regressor as 'good'.
Do you know how can I fix this? Does it exist any other method similar to this one that allows to use this big amount of Data samples?
 A: This depends on what you care about.
For forecast accuracy just use cross validation with the mse of test sets to select features. For inference then I would let any theory you have for the domain get you started and then what you do from there is, quite frankly, up for debate but a full bayesian treatment injecting as much domain knowledge into your priors would be my recommendation.
Variables that are 'linked' begs the question of correlation vs causation.  I will just assume correlation.
For simplicity, you could just toss adjusted r squared and minimize the AIC or AICc or BIC.  They have some nice qualities that r-squared does not.  Minimizing AIC results in a model that approximates the best model from cross validation and BIC approximates 'True' model process if it exists as a subset of your variables (this is almost never the case). All have been shown to outperform R squared and Adjusted R squared on average for model selection in simulated studies from awhile back.
Alternatively, you could use a LASSO regression to do variable selection and regularization. If you use scikitlearn you can just use the LassoCV class and it will handle everything for you.  Variable coefficients will be 'shrunk' towards 0 and many may be 0 and therefore dropped from your equation.  What's left would be variables that were more 'useful'.
And finally, you could just feed it to a random forest and use the feature importance to grab the most 'relevant'.  Definitely a looked down upon approach from a statistics standpoint but it has some nice-ities.  Primarily it will handle feature interactions better than standard regressions so it is pretty useful as plug-and-play approach.
