9
$\begingroup$

I am new to ridge regression. When I applied linear ridge regression, I got the following results:

>myridge = lm.ridge(y ~ ma + sa + lka + cb  + ltb , temp, lamda = seq(0,0.1,0.001))
> select(myridge)
modified HKB estimator is 0.5010689 
modified L-W estimator is 0.3718668 
smallest value of GCV  at 0 

Questions:

  • Is it OK to get zero for GCV?
  • What exactly does it mean?
  • Is there a problem with my model?
  • How can I find the $R^2$ value of myridge?
$\endgroup$
  • $\begingroup$ Change lamda to lambda. I think the sequence you specified was ignored. $\endgroup$ – user36318 Dec 17 '13 at 13:13
20
$\begingroup$

You might be better off with the penalized package or the glmnet package; both implement lasso or elastic net so combines properties of the lasso (feature selection) and ridge regression (handling collinear variables). penalized also does ridge. These two packages are far more fully featured than lm.ridge() in the MASS package for such things.

Anyway, $\lambda = 0$ implies zero penalty, hence the least squares estimates are optimal in the sense that they had the lowest GCV (generalised cross validation) score. However, you may not have allowed sufficiently large a penalty; in other words, the least squares estimates were optimal of the small set of of $\lambda$ values you looked at. Plot the ridge path (values of the coefficients as a function of $\lambda$ and see if the traces have stabilised or not. If not, increase the range of $\lambda$ values evaluated.

$\endgroup$
5
$\begingroup$

The reason you are getting a 0 GCV is because you used:

myridge = lm.ridge(y ~ ma + sa + lka + cb + ltb , temp, lamda = seq(0,0.1,0.001))

instead of

myridge = lm.ridge(y ~ ma + sa + lka + cb + ltb , temp, lambda = seq(0,0.1,0.001))

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.