I was doing some simulation on the Lasso. Particularly, I set p=200 variables, where only the first 5 have non-zero coefficients. I generated a training sample of size n=100. Whatever I do to tune the hyper parameter lambda, it is hard to find a good lambda that do well in both variable selection (only the first 5 variables have nonzero coefficients) and prediction (low prediction error). The reason I observe is that we need to reach a certain value of lambda to leave only 5 nonzero coefficients, however, the estimated 5 coefficients become very small and almost have no effect due to the penalization by the large lambda.
Is there a way that we can manipulate the data to make Lasso work well in both variable selection and prediction?
P.S. I know doing an extra adaptive Lasso step may help a little bit, but is there any way that we can solve this by manipulating the data (transformations) only?