I am given a data set with 1 target variable and 12 features for only 18 observations. My goal is to build a model that has the smallest expected prediction error.
I am allowed to use simple methods such as ridge, lasso, back- and forward selection, PCA. Is there a way in which these methods can be combined.
So far I ran PCA, which based on Cross validation and one standard error rule tells me to select only the first principle component. Looking at the scree plot, it would be more logical to use the first 4 principle components. I think this can be caused by the small sample.
Is using only PCA the best method to model this small amount of observations (interpretation of features is not needed) or can PCA be followed by a lasso on the given Principal components
