We have a dataset with around 20,000 variables and only 200 observations.
Our Naive Modelling:
- We split it into train set (=150 observations) and validation set (=50 observations) and fit Linear regression.
- Results / Train set: As expected, we get $R^2$ as 1 and $MSE$ as ~0 for train set.
- Results / Validation set: As expected, validation set's results are horrible ($R^2$ = -854).
Is there a better way to model (such under-determined system as ours)?