I was reading the paper "Efficient Regularized Regression with L0 Penalty for Variable Selection and Network Construction" in which iterated ridge regression is used to solve L0 penalized regression problems.

It argues that the regular ridge regression solution

solve(crossprod(X)+lambda*diag(ncol(X)), crossprod(X,y))[,1] 

can be solved much more efficiently for the p >> n case using a "dual algorithm" by calculating ridge regression instead as

(t(X) %*% solve(tcrossprod(X)+lambda*diag(nrow(X)), y))[,1] 

I was wondering what is the underlying logic of this, i.e. what are the assumptions and requirements to make this work, and why is this called a "dual algorithm"?

Also, I was wondering what would be an effective way to impose nonnegativity constraints on the fitted coefficients for the p >> n case. For the regular ridge regression

solve(crossprod(X)+lambda*diag(ncol(X)), crossprod(X,y))[,1] 

I can simply substitute the solve with a call to nnls, ie nonnegative least squares, but is there any equivalent way to get nonnegativity constraints efficiently for the p >> n case within the dual algorithm above ?

EDIT i am looking for a solution for a single lambda value so glmnet is no good for me.

  • $\begingroup$ Could you edit your code blocks such that they cover more lines. They are difficult to read for people browsing the site by phone. $\endgroup$ – Sextus Empiricus Sep 14 '19 at 8:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.