If your aim is optimal in-sample performance (wrt highest R-squared), then just use OLS on every available variable. Dropping variables will decrease R-squared.

If your aim is good out-of-sample performance (which is usually what is much more important), then your proposed strategy will suffer from two sources of overfitting:

• Selection of variables based on correlations with the response variable
• OLS estimates

The purpose of LASSO is to shrink parameter estimates towards zero in order to fight above two sources of overfitting. In-sample predictions will be always worse to OLS, but the hope is (depending on the strength of the penalization) to get more realistic out-of-sample behaviour.

Regarding $p > n$: This (probably) depends on the implementation of LASSO you are using. A variant, Lars (least angle regression), does easily work for $p > n$.

If your aim is optimal in-sample performance (wrt highest R-squared), then just use OLS on every available variable. Dropping variables will decrease R-squared.

If your aim is good out-of-sample performance (which is usually what is much more important), then your proposed strategy will suffer from two sources of overfitting:

• Selection of variables based on correlations with the response variable
• OLS estimates

The purpose of LASSO is to shrink parameter estimates towards zero in order to fight above two sources of overfitting. In-sample predictions will be always worse than OLS, but the hope is (depending on the strength of the penalization) to get more realistic out-of-sample behaviour.

Regarding $p > n$: This (probably) depends on the implementation of LASSO you are using. A variant, Lars (least angle regression), does easily work for $p > n$.

If your aim is optimal in-sample performance (wrt highest R-squared), then just use OLS on every available variable. Dropping variables will decrease R-squared.

If your aim is good out-of-sample performance (which is usually what is much more important), then your proposed strategy will suffer from two sources of overfitting:

• Selection of variables based on correlations with the response variable
• OLS estimates

The purpose of LASSO is to shrink parameter estimates towards zero in order to fight above two sources of overfitting. In-sample predictions will be always worse to OLS, but the hope is (depending on the strength of the penalization) to get more realistic out-of-sample behaviour.

So much to the purpose of LASSO. In practice, your strategy (which even got a name: "Leekasso"), seem to be working quite good: Jeff Leek describes a simulation study in his blog:

http://simplystatistics.org/2014/01/04/repost-prediction-the-lasso-vs-just-using-the-top-10-predictors/

Regarding $p > n$: This (probably) depends on the implementation of LASSO you are using. A variant, Lars (least angle regression), does easily work for $p > n$.

If your aim is optimal in-sample performance (wrt highest R-squared), then just use OLS on every available variable. Dropping variables will decrease R-squared.

If your aim is good out-of-sample performance (which is usually what is much more important), then your proposed strategy will suffer from two sources of overfitting:

• Selection of variables based on correlations with the response variable
• OLS estimates

The purpose of LASSO is to shrink parameter estimates towards zero in order to fight above two sources of overfitting. In-sample predictions will be always worse to OLS, but the hope is (depending on the strength of the penalization) to get more realistic out-of-sample behaviour.

Regarding $p > n$: This (probably) depends on the implementation of LASSO you are using. A variant, Lars (least angle regression), does easily work for $p > n$.