I'm a bit confused about the last step in a machine learning assignment. My task is to identify a model which best predicts a response variable.
I've got some data, and initial inspection shows that at least one of the independent variables is linearly related to the response variable. Great!
So I've run some model selection techniques (i.e. best subset, forward & backward stepwise regression, ridge and lasso regression) and I've identified the optimal number of variables to include the model (using cross validation), and the particular variables themselves.
Now do I insert these variables identified back into the multiple linear regression model to estimate the coefficients? And should this be on all the data or just a subset of the data? Or do I just use the regression coefficients estimated from the best subset regression (the method which had the lowest test MSE) ? The reason I'm hesitant to do the latter is I understand that best subset is a variable selection method and is not used for predictions -- is this correct?