I am analysing microarray data in order to build a model for predicting cell proliferation (a continuous variable) based on gene expression (also a continuous variable). There are many more genes than samples (p>n), so I can't use ordinary linear regression methods for variable selection.
For this reason I tried to use least angle regression (LARS), using the lars package in R. I performed cross-validation to determine the number of steps at which the cross-validated mean squared error was lowest, and used this number of steps to build the model.
Out of curiosity, I also built a simple linear regression model using the single most correlated gene. To my surprise, the cross-validated mean squared error for this model was lower than that of the LARS model with more predictors.
I then tried making a LARS model with a single predictor, which was the same predictor that was most correlated and used in the linear regression model. This was an attempt to test that I was building the LARS model correctly, thinking that I should get the same model for both methods using the same predictor, as my understanding was that LARS was a way to select variables for a regression model in cases of p>n, but that the resulting coefficients would be the same for the same set of predictors. However, this was not the case, and the LARS model with the single predictor performed worse in cross-validation than the linear regression model. After checking my code, I don't think that I am building the LARS model incorrectly, I think that the simple linear regression model and the LARS model genuinely give different results with the same predictor.
In all cases leave-one-out cross-validation was performed. I chose this method as I only have 7 samples.
My questions are:
- Why did the LARS model perform worse than the simple linear regression model with a single predictor? To me this suggests overfitting in the LARS model, but performing the cross-validation of the LARS model in order to select the parameter for the number of steps should have avoided this - if fewer predictors in this model had been better, surely the cross-validated error would have been lowest at a lower number of steps?
- Why was the LARS model with a single predictor not the same (worse) as the simple linear regression model with a single predictor?
I am fairly inexperienced in statistics, so easy-to-understand explanations would be appreciated!