1
$\begingroup$

So I've used the Lasso method to fit a 15 predictor multiple linear regression model on the College dataset (ISLR package) with Outstate as the response variable. The problem is that the MSE value obtained is an 8 digit number which seems absurd. Using the forward stepwise method to choose a linear model with 13 predictors I obtained a 7 digit MSE value which also seems too large, but logically it should not be lower than than my Lasso model with 2 more predictors.

 library(ISLR)
set.seed(1)
train.ind = sample(1:nrow(College), 0.5*nrow(College))
college.train = College[train.ind,]
college.test = College[-train.ind,]
xtrain = model.matrix(Outstate ~., college.train)[,-1]
ytrain = college.train$Outstate

xtest = model.matrix(Outstate ~., college.test)[,-1]
ytest = college.test$Outstate
cv.out = cv.glmnet(xtrain, ytrain, alpha = 1) # Fit lasso model on    training data
 bestlam = cv.out$lambda.min
lambda <- 10^seq(10, -2, length = 100)
lasso_mod = glmnet(xtrain,ytrain,alpha = 1, lambda = lambda)

lasso.pred = predict(lasso_mod,s = bestlam, newx = college.test)

MSE = mean((lasso.pred - college.test$Outstate[-2])^2)
$\endgroup$
  • $\begingroup$ Please include your code as text (using code formatting), do not just post a photograph or screenshot (see here, or just note that it is easier for us to copy-paste code than type your script (introducing errors along the way)). $\endgroup$ – Stephan Kolassa Apr 1 '20 at 10:55
  • $\begingroup$ @StephanKolassa Fixed. $\endgroup$ – Pame Apr 1 '20 at 11:06
0
$\begingroup$

I notice you construct a model matrix and remove the intercept. If you do not want a intercept you should specify Outstate ~0+..

Below I do the forward stepAIC:

calcMSE = function(pred,obs){mean((pred-obs)^2)}

library(MASS)
fitAIC = stepAIC(glm(Outstate ~ 0+.,data=college.train))
AIC_MSE = calcMSE(predict(fitAIC,college.test),college.test$Outstate)

AIC_MSE
[1] 3757030

mean(College$Outstate)
[1] 10440.67

sqrt(AIC_MSE)/mean(college.test$Outstate)
[1] 0.1864213

You can see I get a 7 digit MSE but it's ok given that the mean of your response is ~ 10k, remember MSE is the mean square of your error, so a quick approximation shows that it's about 20% off..

I cannot reproduce so your called 8 digit MSE for lasso. Some parts of your code (for example predict(lasso..)) throws an error, so I re run something similar below:

library(glmnet)
xtrain = model.matrix(Outstate ~0+., college.train)
ytest = college.train$Outstate
cv.out = cv.glmnet(xtrain, ytrain, alpha = 1) 
bestlam = cv.out$lambda.min
lasso_mod = glmnet(xtrain,ytrain,alpha = 1, lambda =bestlam)

lasso.pred = predict(lasso_mod, 
newx = model.matrix(Outstate ~0+., college.test))

lasso_MSE = calcMSE(lasso.pred,college.test$Outstate)

lasso_MSE
[1] 3720351

You get a MSE lower than the stepAIC.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.