I was reading the paper "Efficient Regularized Regression with L0 Penalty for Variable Selection and Network Construction" in which iterated ridge regression is used to solve L0 penalized regression problems.
It argues that the regular ridge regression solution
can be solved much more efficiently for the p >> n case using a "dual algorithm" by calculating ridge regression instead as
(t(X) %*% solve(tcrossprod(X)+lambda*diag(nrow(X)), y))[,1]
I was wondering what is the underlying logic of this, i.e. what are the assumptions and requirements to make this work, and why is this called a "dual algorithm"?
Also, I was wondering what would be an effective way to impose nonnegativity constraints on the fitted coefficients for the
p >> n case.
For the regular ridge regression
I can simply substitute the solve with a call to
nnls, ie nonnegative least squares, but is there any equivalent way to get nonnegativity constraints efficiently for the
p >> n case within the dual algorithm above ?
EDIT i am looking for a solution for a single lambda value so glmnet is no good for me.